Central to the problem of most of the states in Africa as a whole and Nigeria in particular is the excessive centralization of the federal system. The centralized federalism denies the opportunity for self-expression, autonomy and by extension prevents avenues for negotiations towards attaining equity and justice. The practice of federalism with a strong center, given the case of Nigeria state has altered the balance between the federal, state and local government. Several responsibilities of the state and the local government have been taken over by the federal government and the implication of this is that power meant to be shared becomes inequitably distributed. This in a way has hindered the meaningful distribution of power, opportunity and resources, which could have improved the well being of the people at the grassroots level. Decentralization would thus be more suitable in emerging democracies as we have in Africa. Similar to this is the defects on the constitution which is meant to define what is right by creating a system of laws whose foundation is hinged on justice, fairness, equality and freedom. At present, the democratization process going on in most African nations promised some ideals, programs aimed at ameliorating poverty and reducing unemployment rate have been put in place, but how would these reformation out live each democratic regimes is the quest for constitutionalism. Given this, the problem of socio-political instability and disorder which are consequences of structural imbalance in the so called democratic system would be effectively resolved if adequate attention is paid to the question of distributive justice. Distributive justice centers on how benefits and burdens rights and privileges, powers and wealth are distributed or shared among the people in a society. This paper, therefore, adopts John Rawls’s theory of justice as its theoretical framework. Specifically, emphasis is laid on the Rawlsian Difference Principles, which prioritizes the demand of social equality over that of liberty and thus would reduce the problem of social inequality and its attended negative consequences. The philosophical methods of conceptual analysis and reconstruction of ideas are employed to establish the relevance of Rawls’s Difference Principle in the quest for decentralization and constitutionalism, which would lead to the attainment of a stable social and political order in contemporary African societies.