What is the best way to stop a stone foundation from crumbing?

I would find an old school mason and see what he recommends. I would also check to see if its crumbling due to moisture or water problems, you may want to put in a french drain if this wall is at the bottom of a hill or where water accumulates. Check your gutters and downspouts too, sometimes just adding downspout extensions prevents water from affecting the foundation. I have to clean my gutters at least 2x a year.

https://www.youtube.com/watch?v=jgU245CbALc lots of videos to give you ideas

There's a few steps and each one combined with the others are necessary. First off the rubble and stone were simply stacked in the old days and then backed with the earth on the outside [ maybe on the inside too if you have a crawl space rather than a basement ]. Goes without saying, gutters as indicated by the previous writer, are essential. Regardless, the first solution is to dig the mortar out and squeeze new mortar into each joint, if there is "old" mortar at all. The second solution is to dig out all around the foundation and waterproof it with foam insulation and spray, "Vulclay" [ kind of like cat litter ] at the base of the walls is a good addition; it absorbs and expands. Actually, I am assuming there is no drain tile around your building, and that too is an essential part of avoiding water seepage in modern homes. I live in a home that is 105 years old. As a Carpenter there is an old joke, "They don't build 'em like they used to, today there are laws."

Had this issue in my 1860's house. We built a "wall of boards" with braces slanted from the floor to them holding them several inches from the wall and poured concrete, making a " new " inside wall. Time consuming and a pain in the butt, but better than a wet basement. It has held up very well for over 15 years now.

From the picture, it appears that the paint is coming off rather than the stone crumbling. With water and over time, the paint, which often has sand mixed in to create texture and enhance coverage, becomes old. It comes off in chunks or flakes off. It's similar to the coating put on stucco houses. From a structural standpoint, it is harmless.