Cool to see that this worked well for someone. Super hard to force the key insight in a problem to magically appear given more time sunk into it. Big weakness of mine honestly, and requires a lot of self-awareness to pull myself out of a problem-solving rut. I like the idea of hacking sleep - do you find yourself priming your mind with the problem before nodding off? Curious how a bedtime wind-down routine factors into how effective this is.

Over years of math undergrad and grad school I tried very hard and was never able to get this to work, so you're not alone. I was able to reliably reproduce hopeful feelings after sleep, but upon investigation the "new leads" were either things I had already tried (and forgotten why they didn't work) or they were the type of imprecise high-level vague direction ideas that were never difficult to generate and still had 99% of the true effort remaining to grind through the details.

While programming Java full-time, I found myself waking in the middle of the night with some big chunk of my brain grinding away on coding issues. Not good.

Reminds me of a similar situation I experienced while working on a big project. I came down with the flu and woke up in a delirious, feverish state at 2am feeling like I was trapped in the codebase and I needed to make all the tests pass so I could escape. It almost felt like my conscious brain had somehow found its way into the unconscious part.

I've had a couple of these.

The first one was programming; I'd been working on a little image editing application, implementing anti-aliased gradient brushes (one colour in the centre and another at the edges, with a fast enough and close enough hack to deal with the jagged edges).

The fever had me hallucinating circles and how to render them for the entire (unbelievably tedious) night.

The other time I'd been playing RA95 for a few hours (I was addicted to that shit) and started feeling progressively worse over the space of a few minutes.

I called it an early night (it was about eight o'clock) and went to bed - then it got worse;

The fever, the headache, and an army of little men and tanks running all over the ceiling.

I guess not strictly on topic, but I once had the dubious idea to read a book about the battle of Stalingrad while having the flu and high fever.

Drifting to sleep and waking up with that book in my head was a ride.

Coding nightmares! I get these too, when working too much. Perseverating on solving intractable problems that don't exist. Once solved it though, and woke up thrilled. That was a good work-night.

Thank you for teaching me the word "perseverate"!! It's a good one!

I agree. It can definitely be useful, but not if I'm stressing too much about it. Then, waking up feels like I was hit by a bus in the night.

Caffeine will do that to you.

I get these as well - with any difficult coding or even SRE problem or any sort of deep logical problem that I find myself working on until the last moment before I go to bed.

I find that doing something else for about an hour before going to sleep helps with this almost 100% of the time. It could be listening to music, watching TV, reading, playing with my kid, almost anything. But giving my head a full hour to rest from a deep problem helps me sleep without ending up in some logical nightmare that only some construct from my problem could resolve (and tends to make worse)

I do the same with the NYT crossword puzzles. The puzzle for a day is released at 10 pm NY time (except the Sunday and Monday puzzles are released 4 hours earlier) the day before, which is 7 pm in my time zone.

I start the puzzle at the end of my day. If I get stuck and cannot finish it before I fall asleep, probably 95% of the time when I take a look again sometime the next day I immediately see answers to several of the things I was stuck on and finishing the puzzle goes smoothly.

Was that because your mind was working on it in the background, or because a reset in state allowed the next attempt to find new paths?

I wonder if it would be possible to make a study to someone distinguish between those, like where the control group gets REM sleep and the experimental group gets some kind of anesthesia. Is there a difference on how well each group does "returning" to a puzzle, compared to working on a fresh one?

Server reset is at 4am and that's when your action points reset. :D

This pretty much describes my method of working as well. Often at the end of the day, when I’m not feeling like I have enough energy to solve anything new, I’ll spend some time just looking over some new problems I’ll have to solve in the next few days. I don’t try to solve them. I’m just “loading them into my head”. The next morning I usually find some potential solutions waiting for me. Thanks, brain! I’ll repeat that process a few times until I have them solved. Unfortunately, while this works great in real life, it doesn’t work for interviews.

> I’ll repeat that process a few times until I have them solved. Unfortunately, while this works great in real life, it doesn’t work for interviews.

Nor does it work when you have a job and a deadline. Or kids.

When does it actually work for anyone these days?

Would work fine for jobs or deadlines, as long as the deadline isn't always in a day or two.

Striving to focus on important problems instead of just urgent problems would work fine with this method.

Perhaps. One would have to dig themselves out of perpetually having a deadline a day or week ago, which is usually the reason one's looking for ways to improve productivity in the first place.

Go meta here. You're trying to solve the wrong problem. Use this technique to get yourself out of this situation.

At companies that set realistic deadlines and don't want to rush, rush, rush everything. They're rare, but some places do understand "slow is fast".

At least in the US the percentage of households with children under 18 has dropped to 40%, and I think you're somewhere in the EU which is closer to 36%.

What? Do you only start working on something the day before your deadline? Do you not plan ahead at all? If you’re always behind the eight ball, scrambling to land whatever you can slap together by the end of the sprint, it’s not going to work. But if that’s the situation you’re in, nothing will work. You’ve got to plan.

It worked fantastically for me at my last job as a lead, and it’s working great for me now as a consultant.

I don't think it needs to be as "structural" as you're describing here. Simply "work on something else if you're stuck, come back to it after a day, two days, or longer" has long been one of my secrets to get stuff done.

I once worked somewhere that allowed you to work only on one ticket/task, before you were allowed to move on to another. Completely dumb policy for so many reasons.

The one ticket only is how it is done at my current job (except in support, which is a whole new hell) and I hate how it actively deters me from using my brain (and time) efficiently.

My previous team's Scrum master kept bugging me because I had multiple tickets open most of the time. As if I were a child that could not manage his own tasks. I left as soon ia could.

I think it was Thomas Edison who said, “Never go to sleep without a request to your subconscious.”

Excellent advice.

I don't even need the coffee - I wake up having solved the problem.

Generally around 2 or 3 in the morning, just when I need it the most.

I think just about everyone has a similar anecdote.

I was stuck on particularly tricky part of a musical piece I was learning - hours and hours of practice and I just couldn't get it down. Went to bed, woke up and was able to nail it within the hour.

Sleep is underrated.

This sounds like me but exactly at the opposite hour - late at night

That being said the pre req being that it's a problem I've already known about (and likely slept on)

I had a college professor suggest this and I 100% agree. I think of it as loading the problem in your brain. Then sleep on it and you will make a lot more progress in the morning than if you had just spent the same total time in one sitting.

I have the same feeling during coding at night.

In the end, a good sleep is what make it different!

a lot of people ditch the coffee for psilocybin

I don’t think it can be compared to caffeine in its efficacy at all with the mushrooms being far more useful, but do note that both caffeine and psilocybin rely on anecdotes for cognitive performance, despite one being able to be studied as a non controlled substance

This reminds me of what I learned about myself during my years spent at the university. I observed that in the morning my brain is better at understanding new concepts. Mornings were the best time for me to practice and improve problem solving, but I tend to remember less details of what I come across. However, at about 2pm my brain appears to switch to memorizing mode, where I struggle with problem solving compared to the morning, but I will remember a lot more of what I read. I structured my learning activity leveraging this observation. Even to this day (am 46) I can feel the same tendency, e.g., if a problem seems somewhat difficult, I just wait until the next morning, if I can, only to find it easy to come up with some solution that seemed out of reach the previous evening. Also, I try to do most of my reading at night (well, life with a family doesn't leave a whole lot of options for timing anyway).

This resembles the oldest self-optimisation-trick I've ever been exposed to: Soak in a topic short before going to bed, avoid distractions before actually falling alseep, and "watch" your brain learn overnight. I think I was aroun 10 when I was told this trick. And to this date, its about the only tangible memory optimisation technique I've ever put to use. Nothing ever came close to its effectiveness.

I have a similar hack but my breakthroughs typically happen on my morning run (after your mentioned good night of sleep). No ear buds, etc. Just a casual pace and giving my mind time and space to work.

This works. It also works fine for programming tasks. If you feel your struggling or making a slow progress on your current task, switch to a different task.

When you come back to the first one your subconscious has usually processed it quite a bit.

Even during an exam, having a micro-nap or going for a walk is beneficial