Following the demise of the Bretton-Woods, increasing number of countries has been opting for flexible exchange rate regimes. Exiting from fixed regimes however is not without costs. Regime transitions have often been occurred in the midst of a crisis, which has considerable economic costs in terms of output contraction and exchange rate depreciation. Given the big number of countries having still fixed regimes and financial markets that are fairly close and expected to be liberalized sooner or later, issue of exiting a peg without incurring crisis is a real challenge confronting these countries. The aim of this paper is to determine the conditions under which orderly exit is possible. The paper employs Binary Recursive Tree and standard regression frameworks. Analysis shows that countries with higher output gap and overvalued real exchange rate, among others, are doomed to exit in a disorderly way. Following their exit, output collapses and exchange rate depreciates considerably. The ill-managed financial liberalization and macroeconomic stabilization programs seem to lay the seeds of instability. An interesting finding is that the conventional strengths of parametric regression analysis can be dramatically improved by utilizing findings of non-parametric BRT technology. Sample contains all countries depending on the data availability, and covers 1975-2004 period. Copyright (C) 2009 John Wiley & Sons, Ltd.