“The financial service industry has an image problem,”market information and strategies magazine Financial Advisor wrote after last week’s SIFMA Private Client Conference. “Earning back trust was a common theme during the one-day conference.”

There was optimism as well, with talk of nobility in financial services, increasing the “standard of care” and “stewardship.” But many acknowledged the damage done by market manipulation, bad actors and poor performance, with one speaker venturing that Generation X is completely soured on investing.

After tales of Goldman Sachs managing directors calling clients “muppets,” can you blame Generation X? Even recent activity would validate its concerns.

Shelved Confidence

A U.S. federal court recently fined Amsterdam-based high-frequency trading firm Optiver $14 million in relation to multiple attempts to manipulate oil prices with the disruptive “banging the close” trading technique out of its Chicago office in 2007. Three Optiver employees received two- to eight-year bans from trading commodities.

So it is no surprise that more than half of Americans do not trust bankers and brokers to do what is best for the economy. We could justifiably be surprised that the number is only as high as 54 percent. Fidelity Investments’ Kathleen Murphy shared that figure from a CNN poll at the SIFMA Conference, adding that the number is up from 30 percent in 1990.

But this isn’t just a matter of holding culprits accountable. It’s also about holding authorities accountable for punishing wrongdoers -- and not-exactly-wrongdoers.

Market Capital Punishment

That initiative suffered a setback recently when U.K. regulators with the Financial Services Authority failed to fine former UBS executive John Pottage £100,000 for not acting fast enough to fix compliance issues within his arm of the firm. The unprecedented attempt to punish senior management for poor oversight could have been a watershed for restoring confidence in the financial services industry.

Questions aside about who defines “fast enough” and whether or not it is reasonable to make an example of someone in the name of restoring confidence, this case telegraphs regulatory intentions. Authorities will surely learn from this defeat, and make similar attempts with different tactics in the future.

And there is something to be said for visible effort.

Louder than Words

“High-profile crackdowns on insider trading seen in the U.S. and U.K. in the wake of the financial crisis have not been mirrored in Japan,” The Financial Times noted Sunday. “Examples of punishment for market abuses are rare.”

Financial services firms could put their clients before themselves, one speaker at the SIFMA conference suggested, even taking on the role of “a good parent” or caring physician.

But just one thing will improve confidence, according to Fidelity’s Murphy.

“Not by asking for it, not by advertising,” she said. “We need to earn their trust by actions.”