When creating monsters based on real-world animals there is a natural tendency to go for creatures which would predate on humans or are otherwise dangerous to us. Snakes, wolves, tigers, spiders, etc. A second natural tendency is to use animals we typically find loathesome (rats, insects, lizards and so on). A third is to use animals which are very similar to us and reflect our primal nature - apes. A fourth is to use animals which strike us as 'cool' in some sense (such as eagles or cats). There is no point dwelling on this rather trite set of observations.
This tendency does, however, leave major parts of the animal kingdom - herbivores, rodents, etc. - unused as sources of fantasy monsters. You don't really come across sheep-headed orcs, giraffe-people or rabbit-men in fantasy games. The reasons for this are equally trite and not worth elaborating on.
Trying to make rabbit-men frightening would pose an interesting challenge. Let alone mouse-men (or moose-men for that matter). People have an image of rabbits as being either like Bugs Bunny or like the floppy-eared pet they used to feed dandelion leaves to as a kid. Rabbits are also natural cowards, and we eat them. They are about the least threatening things you could think of. Yet that means that if you could pull off that coup you could have something genuinely interesting on your hands.
Coming face to face with a rabbit in the wild (as I often do when out and about in the countryside), you're struck by their skittishness. As soon as you see one it will typically have already clocked you and will be watching you with icy terror, torn between the urge to flee into its burrow and its overriding need to eat. If you are far enough away it will tentatively nibble on some grass, constantly on the look out for the myriad dangers (fox, buzzard, dog, human) which could appear at a moment's notice.
Perhaps this is what makes rabbit-men scary: they come cloaked in a thick veil of fear which seeps into your thoughts and saps your will and fortitude, so that pretty soon you have the heart and mind of a rabbit yourself - scared of your own shadow, spooked by the slightest sound. Maybe that's why they are to be feared. It's because they embody fear itself.