When focusing on a problem, I often don't see a possible solution because I haven't relaxed and reconsidered the big picture. When I take a break and stop thinking about the details of a problem, often thoughts about the problem linger in my mind, and as I forget about the details, I'll think more about the big picture than I did while trying to solve the problem. This leads to what I call a "shower moment" (because it often happens when I take a shower). This is common for most people, particularly software developers.
There's more to it though, than simply looking at the big picture. As I'm relaxing and just letting my mind wander, I skip from topic to topic, sometimes thinking about the problem, sometimes about other software, and sometimes about my dogs, family, music, sports, news, etc. Skipping around like this causes me to associate parts of unrelated thoughts with each other. This free association rearranges old ideas in new ways, often taking a pattern observed in one idea and applying it to another idea, sometimes leading to a new perspective on the original problem. (As an aside, one bad thing that TV is responsible for is preventing these free associations. TV is a distraction from this, and forces me to associate ideas in the ways dictated by the show I'm watching. This is okay sometimes, but I need time when I'm neither watching TV nor doing strenuous mental exercises to allow free associations to happen. Some activities, like chores or cooking, are trivial enough that they don't distract.)
Dreaming actually seems quite similar to this free association, except it's more extreme. How often do you have a dream containing actions or situations that seem to be perfectly reasonable, but then after you might wake up you say, "Wtf was that!?" While dreaming, I believe I'm generally doing the same thing; I make random associations between ideas, things, and actions that often are complete nonsense. However nonsensical, they represent associations that my brain has created that could be useful later on, although I'll never know when I'm using them. This may be what causes you to think of something seemingly unrelated to what you're thinking about, and what causes you to think of a new unusual solution to an old problem.
I think dreaming may be very important for our creative abilities. These associations help us think of new solutions for problems, new designs for projects, new art, new music, etc. The more we dream the more creative we are.