The E46 sport seats are an upgrade to the standard ones but still not super comfortable imho. My E46 has the misfortune of being compared to my E60 with the sport seats, so my opinion may be tainted.
E83 seats are a direct swap. The comfort and sport E83 seats are genuinely comfortable. The comfort seats were a popular option on that chassis and will have a circular button for lumbar support. Don't bother with standard E83 seats. The E83 comfort seats don't have as much bolstering as E46 sports seats, but a little more than standard E46 seats.
Seats from an E63 use the same rails but require a bit of modification to the seat belt pretensioner system. If you can find an E63 with sports seats, they are very similar to E60 sport/comfort seats and would be a huge upgrade over any E46 seat, including M3 seats imho. They are more difficult to find and more expensive.
E60 seats are not compatible.
E90 seats can be swapped in with very significant modification to the seat rail system. It has been done, but I wouldn't recommend it unless you enjoy custom fabrication.