VNC specific DES implementation.
Find more info on Vidar Holen dedicated webpage
What VNC actually does
The RFB specification says that VNC authentication is done by receiving a 16 byte challenge, encrypting it with DES using the user specified password, and sending back the resulting 16 bytes.
The actual software encrypts the challenge with all the bit fields in each byte of the password mirrored.
Actual password ("COW"): 01000011 01001111 01010111 (plus five bytes of zero) Key used as encryption: 11000010 11110010 11101010 (plus five bytes of zero)
The actual flipping is, in effect, done in rfb/d3des.c in realvnc.com's unix server. The bytebit array has had its entries mirrored, as the comments mention.
This is no doubt because the most significant bit in 7-bit ascii is always 0. When the DES algorithm creates a 56bit key based on the password, it makes sense to trick it into removing this 0 instead of the the least significant bit which is part of the actual password (otherwise B and C would for authentication be the same character).
See the dedicated LICENSE file for full credits.
noVNC, a web-based VNC that use this DES specific implementation