I would argue that if sex work were destigmatized/decriminalized, more people would become consumers -- there are plenty of people who might choose to visit a prostitute if suddenly it was a safe, legal option. And, as Amber said, there are barriers that prevent people from becoming sex workers that are far beyond stigma/legal concerns (it's just not a job everyone can/wants to do, frankly), so the supply of sex workers would probably still be limited, keeping prices as high as demand will allow.
But, hey, why don't we just look to the Netherlands? Sex work is decriminalized there, and seems to be destigmatized, at least in comparison the the United States. Does anyone have statistics on the profitability of sex work there?