Khan rejected the proposal on the grounds that it would provide "very limited public benefit" to the City of London and that it was not the "world class architecture that would be required to justify its prominence".
The viewing platform and gondola ride attraction would not have been free to enter for the public.
The axed tower was due to be built alongside the RIBA Stirling Prize-winning 30 Mary St Axe – known as the Gherkin, which was also designed by Foster + Partners.
Tulip rejected over concerns for skyline
Plans for The Tulip had originally been approved by the City of London, but the mayor has now rejected the application.
"The mayor has a number of serious concerns with this application and having studied it in detail has refused permission for a scheme that he believes would result in very limited public benefit," said a spokesperson for Khan.
"In particular, he believes that the design is of insufficient quality for such a prominent location, and that the tower would result in harm to London's skyline and impact views of the nearby Tower of London World Heritage Site," the spokesperson added.
Khan also felt the design would "result in an unwelcoming, poorly-designed public space at street level."
Tulip team to "consider potential next steps" for project
Foster + Partners plan included a pocket park at the base of the concrete stem of the tower, a series of rotating glass gondola rides on the sides, and educational facilities for local school children located somewhere in the bulb-shaped glass tip.
"The Tulip Project team are disappointed by mayor of London's decision to direct refusal of planning permission, particularly as The Tulip will generate immediate and longer-term socio-economic benefits to London and the UK as a whole," a spokesperson from the project told Dezeen.
"We will now take time to consider potential next steps for The Tulip Project."
When the planning documents were submitted a report from Deloitte estimated the building, with its observation deck and panoramic restaurant, would create £1 billion of "monetised value" by 2045.
However, the mayor's office had warned that 305-metre-tall viewing attraction was in breach of its planning guidelines for the city back in January 2019.
In recent years numerous skyscrapers have been built in London, including the nearby Walkie Talkie, which won the Carbuncle Cup in 2015 and RSH+P's Cheesegrater. Many more tall towers are planned for the city with a record 541 skyscrapers proposed for the London. These include a pair of towers designed by Zaha Hadid Architects and Skidmore, Owings & Merrill's The Diamond, which is set to rise to 56 storeys.