Is it possible to use a "headless" mac with just:out?

Yes, this is possible, but please keep in mind the following recommendations:

When using a headless system, it is highly recommend not to change the resolution of the virtual display. Doing so will corrupt the output of just:out, showing non moving black blocks randomly spread over the whole output. In this case only a restart of just:out will fix the problem.

Once just:out is started with a display connected to the graphics card, it is not possible to change the resolution or to remove the display any more. In some cases, even turning the display off can make the outgoing signal corrupt. In such cases you must restart just:out to fix the issue.

We have seen screen-sharing (remote) applications change the screen resolution. Please test your solution before going on air.

Even it is technically possible to run the Mac headless, we strongly recommend to use at least a simple HDMI monitor emulator hardware in case no display should be used. Please read the admin handbook for more information.