An additional parcel of more than 60 acres, less than a mile from the Blue Creek terminal, will also be sold.
The total consideration to be paid by the port authority is $US25 million ($A27 million).
The company also agreed to amend the terms of the coal handling agreement in accordance with the port authority handling the company's coal exports through the McDuffie coal terminal in Mobile.
As part of the amendment, the port authority has agreed to a long-term extension of the existing coal handling agreement and to make improvements to the McDuffie terminal to ensure Walter Energy is able to export the coal it will produce when the company's Blue Creek mine is completed.
Alabama State Port Authority director and CEO James Lyons praised the deal as key to the authority's growth in container and intermodal areas.
"Strategically, these acquisitions will help us expand our container business," Lyons said.
Last month major meetings were held with local residents and shareholders in opposition to the terminal over concerns of an increase of coal dust.
The decision to sell will no doubt appease the controversy but comes as another blow for Walter Energy, which recently announced it would idle two mines in British Colombia, laying off more than 700 workers.
