The .41 Magnum is a good caliber and often overlooked. It's more than the .357 Magnum and less than the .44 Magnum. The .41 comes into its own with handloads of different velocities and bullets. The standard commercial loadings are powerful but don't really capture the .41's true capabilities.
Of the three calibers: (1) .357 Mag, (2) .41 Mag, and (3) .44 Mag, the .41 comes in a distant third. Your choice should be tempered by several things (aside from purchase price). The .357 Mag is the most popular of calibers and you can fire the .38 Special cartridge in these guns. The .44 Mag can also fire the .44 S&W Special round. There is currently no other cartridge compatible with the .41 Mag however.
I have one .357 Mag, a Colt Trooper Mk III with a long 7-1/2 inch barrel. This gun shoots like a rifle and is just fine for me. I've shot the .44 Mag in a 4-inch barrel and my feelings at the time were: "When you put wheels on this cannon, come back and see me." I haven't shot the .41 Mag, so I won't comment on it.
One more thing, if you're going to go with a magnum of any kind, then to get the maximum out of it, you have to have a long barrel. The reason is called "sight radius." The longer the barrel, the longer the sight radius from rear to front and the more accurate it is likely to be. Of course, you can mount an optical sight on your pistol for greater accuracy. Be aware that the longer the barrel the more weight you have on the end and the more ungainly the pistol may be for you to shoot. However, it really comes down to what you want to do with the pistol.