There is no right answer here, as all Ports are a little different and everyone has different tastes or ideas when they think a Port is ready to drink. However, here are some basic guidelines to help you out.
Vintage Ports typically need at least 15 years to start reaching maturity. The top Vintage Ports can easily last 30-100+ years if stored properly.
Late Bottle Vintage Ports that are filtered are not meant to be aged. So there is no reason to do so. Unfiltered LBV’s generally will start showing their best at around 10+ years of age. Generally, they are not designed to be aged beyond 20 years, with a few exceptions.
Tawny Port with an Indication of Age is not meant to age in bottle. This type of Port group is usually best when consumed closer to the date of bottling.
There is great debate about Colheitas and whether or not they improve after being bottled. Some think they are at their best within a few years of bottling and some think they can age and improve in bottle. You’ll have to make up your own mind on that one!
Basic Ruby and Tawny Ports, as well as basic White Ports should not be aged at all.