Named Sydney Modern, the SANAA-designed building will be built as an extension to the gallery, which overlooks Sydney Harbour near the centre of the city.
Designed to contrast the gallery's existing 19th-century neo-classical buildings, the new standalone building will be formed of a series of rectangular pavilions that step down a hill towards Woolloomooloo Bay.
The two buildings will be separated by a square described by the gallery as "a public art garden".
The interlocked pavilions will be connected by a large, multi-level atrium that will have views towards Sydney Harbour through glass walls.
In total, the extension will contain 7,000 square metres of gallery space and include two galleries that are each over 1,000 square metres with 5.5-metre-high ceilings.
The building will also contain a range of smaller galleries, learning studios for children and school groups, a dedicated studio for creating multimedia works and a rooftop art terrace.
Another major gallery space will be under the building, as the structure will be built on top of a world war two oil tank.
This space will be converted into a 2,200-square-metre underground art space, which will be used to host large-scale commissions and site-specific performances.
The project, which is scheduled for completion in late 2022, will be Japanese architecture studio SANAA's first in Australia.
"We hope the Sydney Modern Project will be a special place for visitors to experience the shared joy of art and ideas in this unique and beautiful setting," said Kazuyo Sejima, co-founder of SANAA.
"This is our first building in Australia, and we are delighted it is being realised."
SANAA is a Tokyo-based architecture studio that was founded by Kazuyo Sejima and Ryue Nishizawa in 1995. The studio created a cloud-shaped port terminal building for Naoshima island and designed the 2009 Serpentine Gallery Pavilion in London.