Little Snitch 4 is using a Network Kernel Extension to perform its network filtering. Apple has discontinued the support for this type of kernel extensions in macOS Big Sur. The system therefore refuses to load the unsupported extension.
Little Snitch 5 has been updated to the new Network Extension technology instead, which is fully compatible with macOS Big Sur.