Rallis was the son of former Greek Prime Minister Dimitrios Rallis, and he came from a family with a long tradition in political leadership. He studied law at the National and Kapodistrian University of Athens, as well as in France and Germany. Upon his return to Greece he became a lawyer. In 1905, he was elected as a member of parliament for the first time; he remained in parliament until 1936, when democracy was abolished in Greece by the 4th of August Regime of Ioannis Metaxas.

Rallis originally belonged to the Greek conservative and monarchist People's Party. As a member of this party he served in various administrations as:

• Minister of the Navy (4 November 1920 – 24 January 1921). Under Prime Minister Dimitrios Rallis, his father.
• Minister of Economics (August 26, 1921 – March 2, 1922). Under Prime Minister Dimitrios Gounaris.
• Minister of Foreign Affairs (November 4, 1932 – January 16, 1933). Under Prime Minister Panagis Tsaldaris.

After the victory of the People's Party in 1933 Greek legislative election, he served the new government under Tsaldaris from various posts. In 1935, he had a disagreement with Prime Minister Tsaldaris, the leader of the People's Party, and at the ensuing 1935 Greek legislative election he campaigned with Ioannis Metaxas and Georgios Stratos as a candidate of the Freethinker's Party, but he failed to win election.

Greece was in a time of great political instability and new elections were held, the 1936 Greek legislative election. This time Rallis joined with Georgios Kondylis and Ioannis Theotokis and he was elected. Parliament was fractured, with the Liberal Party under Themistoklis Sophoulis having a one-seat majority and the opposition divided between monarchists and Communists and every philosophy in between.

When the Metaxas dictatorship was declared later that year, and parliament was dissolved on August 4, 1936, Rallis expressed disapproval of this political coup, despite his personal friendship with Metaxas.

After the successful German invasion of Greece in the spring of 1941, Rallis was the first eminent Greek political figure to collaborate politically with the German occupying forces.^[1] The Germans hoped that Rallis would gain some support from the pre-war Greek political elites, that he might be able to restore order to the country and that he could form an anticommunist front against the Ethniko Apeleftherotiko Metopo (EAM) and the Ethnikos Laikos Apeleftherotikos Stratos (ELAS).

EAM was the main movement of the Greek Resistance and had been initially formed by an alliance of Communist Party of Greece, the Socialist Party of Greece, the Greek Popular Republic and the Agricultural Party of Greece. ELAS was its military arm. Since anti-communism served as a common ground between the Liberal Party and the People's Party, the idea of a united front seemed plausible.

Rallis changed the ministry council and was instrumental in creating the so-called "Security Battalions"—collaborationist paramilitary groups equipped by the Wehrmacht and dedicated to the persecution of resistance groups (mainly ELAS). Being more experienced in politics than his predecessors, a general and a German-speaking doctor, he was more respected by the Germans and proved more effective against the resistance movements.^[2]

All three administrators during the occupation (Georgios Tsolakoglou, Konstantinos Logothetopoulos and Ioannis Rallis) presided over what was in effect a puppet government (1941–1944) completely subordinate to the Nazi occupation authorities. Thus, they all failed to prevent the Nazis from imposing heavy "reconstruction" fees on Greece, paid eventually by the confiscation of crops and precipitating a terrible famine that according to the Red Cross, cost the lives of about 250,000 people (mainly in the urban areas of the country).^[3] However, the Germans had no deliberate plan of starvation of the Greeks.^[4]

After the liberation of Greece, Rallis was sentenced to life imprisonment for collaboration and treason. He died in prison in 1946.

Ioannis Rallis's son Georgios Rallis became prime minister during 1980–1981. In 1947, Georgios published a book entitled Ioannis Rallis Speaks from the Grave, which consisted of a remorseful text written by his father during his imprisonment. A womanizer, during his tenure as Quisling prime minister he had an affair with stage actress Eleni Papadaki. His relationship caused sensation in the Athens of the time, fueled by the antagonist left-wing press.^[5]

• List of foreign ministers of Greece

• Georgios Ghikas, Encyclopedia Πάπυρος-Larousse-Britannica, vol. 51, entry Ioannis Rallis, Athens: Πάπυρος, 1992.
• Mark Mazower, Inside Hitler's Greece. The Experience of Occupation, 1941-44.(Greek translation), Athens: Αλεξάνδρεια, 1994(1993).