Ennui is the French word for boredom but, in literature, it has a somewhat broader meaning. There is no obvious English word for it. The Italians use noia. There is related concept in German which can be called Weltschmerz or even Angst. Latin has accidia. Russian has Тоска (toska). Turkish has Hüzün. English words would include boredom, lassitude, world-weariness, malaise, melancholy, alienation and even anomie. It essentially means a feeling of detachment from the surrounding society and its mores, often due to living in a period and/or place where things are stagnating, often coupled with a general boredom with everything and everybody. It is certainly not a twentieth century phenomenon. Maria Edgeworth wrote a novel in 1806 called Ennui about the boredom of the upper classes. The Romantic poets and the French nineteenth century poets are just two groups who were familiar with it.
In the twentieth century, it came to be associated with the existentialists in France, where it came close to what is called alienation in English, meaning that people only responded to external stimuli. The classic case is Meursault in L’Etranger (UK: The Outsider; US: The Stranger). However, Moravia had been there first with his Gli indifferenti (The Indifferent Ones; The Time of Indifference). He later wrote a book called La Noia, translated as (The Empty Canvas; Boredom). However, the concept has been a key theme of literature for a long time and certainly was in the twentieth century, from Kafka to Salinger and beyond.
David Foster Wallace, in his The Pale King, has a fascinating account (pp 383-385) on the origins of the word bore which, as he states, was, according to the Oxford English Dictionary, first used by the Earl of March in 1776. DFW states that the Earl of March first used it in reference to a French peer. In fact, he did not but said Augustus Hervey and Lord Cadogan are in a long bore. It was, in fact, both Gilly Williams (real name George James Williams but always known as Gilly) in the same year and the Earl of Carlisle the following year who used the term in relation to a French peer. Interestingly the Earl of March in 1776 was a title held by Charles Lennox, Duke of Richmond (who also held several other dukedoms and would therefore almost certainly be referred to as the Duke of Richmond and not as the Earl of March.) As for bored, the Oxford English Dictionary says it was the Earl of Carlisle who first used it in 1768 (I pity my Newmarket friends, who are to be bored by these Frenchmen). DFW’s interesting point is that the word effectively did not exist in English, at least in written texts, till 1766. Boring and boredom do not appear till the middle of the 19th century (Dickens’ Bleak House is the first written use of the word boredom.) John Evelyn says (in 1667) We have hardly any words that do…fully express the French naivete, ennui, bizarre, etc. and Berkeley used the word ennui as a synonym for boredom in 1732.
DFW says other words were used – lethargie/lethargy (first used in the mid/late fourteenth century by Chaucer and Wyclif) and melancholy (first used in the late fourteenth century in the translation of the French romance Guillaume de Palerme (William of Palerne) and by Chaucer, though Gower in his Confessio Amantis of 1390 was the first to use it in the sense of ennui. He also suggests saturninia, a word not recognised by the OED or, as far as I can tell, anyone else, at least with this sense. Saturnine (noun: saturninity), meaning gloomy, was first used in the late fifteenth century, though the noun only in 1903.
Books on ennui in literature
Elizabeth Goodstein: Experience Without Qualities: Boredom and Modernity
Sean Desmond Healy: Boredom, Self, and Culture
Reinhard Kuhn: The Demon of Noontide: Ennui in Western Literature
Patricia Meyer Spacks: Boredom: The Literary History of a State of Mind