Of course not, because the law does not favor forcing people to remain unemployed. But that said, the restrictions you are under depend on what exactly the agreement says, what exactly you propose to do, the state you are in and the extent to which the company is prepared to assert its claims.
For example, California does not permit enforcement of "non-compete" agreements except in extremely limited circumstances, such as if you sold a business to the company that hired you. But it does respect the non-disclosure of trade secrets provisions typically built into many non-competes.
If you start to disclose truly propreitary information, or compete with them head on and seek to take away former clients, that's most likely to get them riled up. If the company has a hard=asss policy, I urge you to consult with a lawyer BEFORE you start the new business, so the lawyer can advise you of what to be wary of, if anything. Being forearmed is wise.