Cuomo is aiming to create a new public/private partnership with a developer to expand the existing station and build a new hub across the street in a massive former post office and mail-sorting facility.
The developer would also control retail spaces in the facilities and gain substantial development rights for new office or residential space above or near the former postal building.
The governor is committing $325 million (£222 million) in government funds for the project, but the total project is expected to cost $3 billion (£2 billion).
The station currently handles 650,000 daily travelers, though it was only designed to handle 200,000 people a day.
Penn Station, as it is commonly called, is located underneath Madison Square Garden, Manhattan's largest arena. It serves Amtrak, the US national railway, New Jersey Transit, which carries passengers from the west, and Long Island Railroad, a commuter system that brings riders from the east.
The underground station is cramped and dangerously overcrowded with low ceilings and poor circulation.
"Penn Station is the heart of New York's economy and transportation network, but it has been outdated, overcrowded, and unworthy of the Empire State for far too long," said Governor Cuomo. "We want to build Penn Station to be better than it ever was, and that is exactly what we are going to do."
The plan would relocate Amtrak into the Farley Post Office Building, which is located across Eighth Avenue from Penn Station, creating 210,000-square-feet (19,509 square metres) of waiting room with retail space. Moving Amtrak out of the existing station will allow more space, simplified circulation, and expanded corridors for New Jersey Transit and the Long Island Railroad.
The two facilities would together be known as the Empire Station Complex.
A theatre located under Madison Square Garden could also be removed, creating a new entrance that would allow light into the existing terminal.
The Municipal Art Society, a civic group that has advocated for the relocation of Madison Square Garden and the construction of an entirely new Penn Station, organised an ideas competition to reimagine the complex. SHoP, SOM, Diller Scofidio + Renfro, and H3 Collaborative all put forward proposals.
The society supports elements of Cuomo's plan but also believes it does not go far enough.
"We echo the Governor's conviction that Penn Station is in need of urgent and ambitious change," said Mary Rowe, executive vice president of the society.
"However, in the long term, these improvements won't be enough to fully address Penn Station's severe overcrowding or meet the growing needs of its rapidly developing neighborhood and our regional economy," she added.
The governor's goal is to receive expressions of interest within 90 days and to break ground by the end of the year.
A pair of prominent New York developers, Vornado and Related Companies, had been tasked with redeveloping Penn Station along similar lines for more than 10 years. They had been seeking an anchor office tenant but had not moved forward with construction to date.
Cuomo's announcement will either jumpstart their efforts or see them replaced. Extell and Brookfield, two other large real estate companies, have already indicated possible interest, according to the New York Times.
The original Penn Station opened in 1910 and was designed by the Beaux Arts architects McKim, Mead & White. One of New York's grandest structures, the building was demolished in 1963 to create the current facility. The ensuing controversy helped fuel the preservation movement in New York and the passage of the Landmarks Law in 1965.
Images courtesy of the office of governor Andrew Cuomo.