I have an iPod, not a shuffle though, i'd imagine they'd work basically the same though.
When you first get it, it won't show up in My Computer as a disc drive until you enable it to be used as a disk through the options in iTunes, if iTunes is closed and you connect the shuffle via usb iTunes should open automatically, and open your device's library thingy in the window, or ask you if you want to update your iPod, if none of that sounds familiar then, I'm assuming you're on XP, does windows pop up anything in the system tray when you connect it? any messages at all? when mine is connected it puts the 'safely remove hardware' icon thing in the sys tray.