About fallback mechanism.
Most popular encoders, media containers and streaming services stores videos as YUV, not RGB. You are using "x264" encoder - it cannot record RGB-like pixel formats. The ".mp4" media container cannot store data that uses RGB-like pixel formats (".mkv" can) - this is by ISO standard for MP4 container. See ISO/IEC 14496-12 for details.
Maybe you want to ask why OBS instead of rising the error attempts to record at least "something"? There are many factors:
- similar fallback mechanism in other utils (re-encoding with FFmpeg - it chooses best compatible pixel format even when format is not specified);
- streaming to popular platforms (no RGB support, only YUV 4:2:0, because RGB-like or YUV 4:4:4 requires a lot of bandwidth);
- developer about custom output: "Video format handling has also changed; it will now attempt to use the closest format to the current format if available for a given video codec." (source: https://github.com/obsproject/obs-studio/commit/d3f92ca5d6bea05f9659cec7524f2a34d03e8c16 was in OBS v0.8.0 long time ago, and there is other examples from other developers: https://github.com/obsproject/obs-studio/pull/3439 that protects this fallback mechanism in other parts of OBS);
- my own opinion: users complains (it doesn't work! fix it! *ing program! I cannot stream! I streamed yesterday but now there is error! etc).
If you only want to know the answer to the question "why 4:2:0 and not 4:4:4?" then here is the code:
OBS Studio - Free and open source software for live streaming and screen recording - obsproject/obs-studio
github.com
This is very simple algorithm "IF not valid THEN...".
When invalid pixel format specified for the x264 encoder, then it will be substituted with NV12 (4:2:0).