There may be an additional source term of people who were unemployed but not poor in year N-1. But since the number of people in poverty remains roughly stable over time, this source must also be balanced by a sink of people who were poor in year N-1 but not poor in year N. I.e., for every person who becomes unemployed and enters poverty, there must be another person who leaves poverty (the exact balance varies a bit from year to year). Numbers on this are somewhat harder to come by (most surveys don't track the movement of individuals throughout categories).
(Again, rough numbers, these fluctuate a bit from year to year. )
This has the effect of juicing the BLS statistics over time by reducing the unemployment by definition. (i.e. if you've been unemployed long enough, you're no longer considered a part of the labor force, regardless of what efforts you may be making to find a job.)
"Persons are classified as unemployed if they do not have a job, have actively looked for work in the prior 4 weeks, and are currently available for work."
""Discouraged workers" are a subset of the marginally attached. Discouraged workers report they are not currently looking for work..."
A discouraged worker is one who has looked for work at some point in the last 12 months, but is no longer looking because they believe there is no work to be found. (and are often correct in aggregate)
Where the BLS further skews the statistics is that discouraged workers who have been discouraged longer than 12 months just drop from the labor force completely, they are a hidden statistic. (I believe this definition was changed under the Clinton Administration)
They might have had good reasons for assuming they would not get a job, for example. Or maybe the searched 40 weeks in vain in the previous year and didn't have it in them anymore.