I migrated from AutoCAD to SolidWorks a couple of years ago, and have also used Pro/E in the past. When I made the decision to move to SolidWorks for the company I am with now, I looked at Inventor, but I don't know much about it specificly. But, moving from one 3D design package to another is not difficult in my opinion. You've got the basic skills and your mind is geared to think 3D. What you have to learn is 1)terminiology differences, 2)command locations, 3)what SolidWorks can or can not do compared to what you did with Inventor.
If you are a quick learned and do not require a lot of one on one training, jump into it. Solidworks probably will not be a problem for you. If you need a little instruction, there are plenty of resources available (this news group, SWX VAR training, books etc.).