One thing I sometimes do for fun is sit in front of my computer, try to completely blank my mind, and then just start writing sentences - channelling my subconcious directly onto the keyboard. I'm going to try it now:
"There is a man lying on the floor and there is a keyboard and a monkey in the room."
But you see there the problem: I just blanked my mind and started typing. But the word 'keyboard' came to me, evidently because I had just written it in the previous sentence. Keyboards were present in my unconscious mind. And when I tried to come up with something 'at random', keyboards are part of what came out. Oddly, when I wrote the sentence, the image of a musical keyboard - a Casio keyboard - flashed into my brain, rather than a computer one. The unconscious works in strange ways.
You see my point, though, which is that your brain can produce unexpected, surprising, and interesting results when you free-associate, or when you try deliberately to void your mind of all influences and just give yourself free reign to blaaaaaaaah. But these results are not necessarily as unexpected as you might think: the things that are in your brain are there for a reason, because they've been taken in and processed. This means they're always imminent, in the background. You're not coming up with things at random - you're reaching into your mind and pulling something out. You may be reaching blind, but it's not quite the same thing.
In the same way, the NPCs and situations that you come up with on the fly may be spontaneous and surprising and unexpected, but I bet if you really thought about it you would find similarities with people (real or fictional) that are known to you or at least seen, or situations that you have been in or heard about or read about.
Now, you might say the same thing about random tables that you, yourself, have created. If you've written up a d100 table, let's say, then everything in it has come from your mind. So when you roll a d100 and consult that table, you're not really getting totally unexpected results, because you know what's in the table, and after all everything there is from your mind, so what's the difference?
I think the difference is that the result on the d100 table is not contingent on recent input. When reaching into your subconcious mind on the fly to pull out an NPC, let's say, you are probably going to be - unintentionally - generating an NPC on your mind who is influenced by what has happened in the last 10 seconds, last 10 minutes, last 10 days. It might not be immediately apparent what that influence is; it might only be at the level of a Casio keyboard because the word keyboard cropped up a while earlier. But there will be something. On a d100 table, on the other hand, the results will be disconnected from recent input and will therefore have more of a capacity to be genuinely weird and surprising, genuinely dislocated from the situation, and thus genuinely more potent and with more potential for being interesting.