This is a column about the Port Authority of New York and New Jersey, but you won’t read a word in here about the lane-closing scandal in Fort Lee, N.J. This is about another scandal, one that has been going for on so long that people don’t even think of it as scandalous. Indeed, it involves no illegality whatsoever. But that doesn’t mean it isn’t a scandal.

The Port Authority is supposed to manage — and improve — important parts of the transportation infrastructure of New York and New Jersey: airports like John F. Kennedy Airport, bridges like the George Washington Bridge, and terminals like the Port Authority Bus Terminal.

And, in fact, all of these need improving, especially the bus terminal, which is 64 years old and thoroughly outmoded. The steep $13 toll that drivers pay to cross the George Washington Bridge, for instance, is supposed to help pay for infrastructure improvements.

For decades, however, at least some of that money has been diverted to real estate — specifically, the World Trade Center, which the Port Authority originally built in the late 1960s and early 1970s, and then subsidized for the next several decades, as the Twin Towers languished under its stewardship. It finally exited the business in the summer of 2001, by signing a 99-year lease with Larry Silverstein, the developer.

Which, of course, was only weeks before the terrorist attacks on Sept. 11. Since then, the Port Authority has dived back into real estate, pouring at least $7.7 billion rebuilding the area around Ground Zero. Some of that money went for the 9/11 memorial and museum. But some $4 billion went to an over-the-top PATH station. And another $3.3 billion has gone to build One World Trade Center — which used to be known as Freedom Tower, and, at a symbolic 1,776 feet high, is now the tallest building in the country.

Whether or not building commercial skyscrapers was the right way to rebuild Ground Zero, what can be said for sure is that the Port Authority has shown, yet again, that it doesn’t belong in the real estate business. One World Trade Center is the most expensive high-rise building ever built in America, and it is costing the Port Authority a fortune. Only 55 percent of its 2.6 million square feet has been leased, and most of that is at a significant loss. Meanwhile, 4 World Trade Center, which was developed by Silverstein, has only 60 percent of its space leased. As The Wall Street Journal pointed out recently, between the two buildings, there is more than 2.5 million square feet of unleased space at Ground Zero.

So why in the world would the Port Authority be willing to back another $1.2 billion in loans to help Silverstein build 3 World Trade Center? Yet on Wednesday, that is exactly what the Port Authority board is supposed to vote on.

Silverstein needs the loan guarantee for a simple reason: The market is saying that, with all that empty office space, this is not the time to be building another skyscraper downtown. He has, so far, found one tenant, but banks are insisting that a higher percentage of the building be preleased before the construction of the building will get financing. So Silverstein has turned to the Port Authority instead to be his funder of last resort.

And not all that long ago, it would have been a safe bet that the Port Authority would have gone along. Indeed, the vice chairman of the board, Scott Rechler — a realtor himself — has said that “it’s part of our mission to finish it.”

But this time, somebody on the board has finally stood up and said, “Enough.” That person is Kenneth Lipper, an investment banker and a former deputy mayor of New York, who was appointed to the Port Authority board last year by Gov. Andrew Cuomo of New York.

“There is simply no reason for the Port Authority to step in,” he told me on Monday. “The private sector is appropriately saying, ‘Not now.’ ” But he also had another objection, one that heralds back to the original purpose of the Port Authority. “Our role is to develop the transportation infrastructure of this region. We have more infrastructure needs than we can finance through our revenue base. As a result, we are triaging necessary transportation improvements to finance what will be an empty building.”

Always in the past, the commissioners have voted unanimously to approve ventures like the Silverstein deal; it was the way things worked at the Port Authority. That’s one reason these expenditures have seemed less outrageous than they really are: there was no opposition. This time, however, there is going to be an actual debate. And if, after that, Silverstein gets his loan guarantees, well, there will finally be no doubt that a scandal has taken place.