If a race had sussed fusion power then their energy provision wouldn't be much of a problem. We're close to it ourselves and we've only been at this sciency thing for a few hundred years. If you had races that were millions of years old and scattered around the cosmos, then it's likely we couldn't even comprehend their technology and they'd probably know a bit about reality
There are also less exotic means of getting to very high velocity. Take a big rock - or preferrably a few and fire rockets and head to somewhere big like Jupiter or the sun. Sling shot around it and point it towards the Earth. No rockets now required. Sling shot around the Earth and point it to whatever destination you want. This rock - or preferrably a few of them need manouvering thrusters from time to time, but don't slow down. Then you get a rocket (Preferrably from the moon) and use the materials there for fuel. Fly up to the rock and catch it up and land on it. Refuel it's thrusters and repeat and repeat. They are actually working on this idea now - basically a very, very fast and very, very cheap way of getting around the solar system and perhaps even further.
This is the idea here..
https://www.youtube.com/v/pUjnRW-igtk
Travelling around our solar system using fusion would still take a long time, Voyager 1 for example has reached a speed of 61,500 km/h and has taken 45 years to reach the edge of the heliosphere.
So travel around our solar system is potentially possible, using slingshot methods or solar sails for example.
The Milkyway itself though is 125,000 light years across. So roughly 9.5 trillion KM x 125,000 across.
Travelling at 99.9% the speed of light means it would take over 250,000 years to make a return trip.
You also have relativity and the effects of time dilation at those speeds to consider.
Further afield, for visitors to come from Andromeda then they have a 2,480,000 light-year journey to undertake before they even consider traversing the milky way.
Then you get into the realms of pure sci-fi with warp travel, wormholes and zero point energy.
All wildly theoretical at present.
If aliens were opening wormholes and bending space time in our local vicinity I think there might be a bit more evidence of it than blurry photos, video and Joe Rogan podcasts.
I'm not saying its 100% impossible, simply that its very unlikely when all factors are considered.
For humans, if we can avoid self destruction, then the future is possibly some sort of Dyson sphere structure to harness the sun's energy for as long as possible, with tiny world-lets orbiting its white dwarf remnants.
Possibly even a colony to the very nearest stars.
In the end though, whatever we do entropy appears to win.