James Ivory was educated at St Andrews and Edinburgh and taught mathematics in Dundee. He became manager of a flax spinning company and then, in 1804, became professor of mathematics at the Royal Military College, Great Marlow (which became Sandhurst), together with William Wallace. Ill-health caused his resignation in 1816; and, for the rest of his life, he lived simply in London.
Ivory was elected a Fellow of the Royal Society in 1815. His difficult personality led him to quarrel with many of the British scientific establishment. Nevertheless, he was awarded the Copley Medal (1814) and the Royal Medal (1838) of the Royal Society of London, he was honoured by many foreign scientific societies, and he received a knighthood in 1831. In 1838 Ivory gave the Bakerian lecture to the Royal Society, entitled On the theory of astronomical refractions.
Ivory and Wallace were early supporters of the work of the French analysts, especially Lagrange and Laplace. Ivory's critical commentary of Laplace's Méchanique céleste was praised by Laplace. Ivory wrote several articles for encyclopaedias, including the influential Equations in Encyclopaedia Britannica. But his main research concerned applications of mathematics, most notably: the gravitational attraction of ellipsoids, the shape of self-gravitating rotating fluid bodies, the orbits of comets, and atmospheric refraction. His work on the ellipsoidal equilibrium configuration of self-gravitating fluids was an extension of that of Laplace, and it influenced the achievements of Jacobi and Liouville which followed.
Ivory was knighted for his contributions to science.