Back in the dark ages (the 1920s until the 1950s) a typical oil or gas furnace was just a large metal box with a burner and a pilot light. The pilot heated a thermocouple which generated electricity to turn on the fuel valve when the thermostat sensed the temperature was low. But, we are in the modern world now and furnaces typically have blowers and such that need electricity. So, what you need is electricity to run your non-electric furnace.
If you have a source of 120 volts AC, all you need to do is supply it to your furnace. If you have a big generator for your whole house you are home free but, what if you don’t? Well this video offers an easy to understand way to solve the problem for about $5. You can then use a small power station or even a power inverter that runs off your car’s 12 volt electrical system to get your furnace up and running.