Mac OS Tahoe 26.6 unable to connect - 'Protocol Error: key length too large'

Hi,

 

I just upgraded my Macbook to Tahoe 26.6 and I'm no longer able to connect to it, I get an popup stating 'Protocol error: key length too large'. I've tried disabling the authenticate using SSO and smartcard/cert options, and also changing the encryption setting to either prefer off, or always maximum.
There were a lot of security fixes in this MacOS release so it's likely something has changed with the supported encryption methods?

Debug logs show the following:
<15> 2026-07-29T08:40:33.510Z **** vncviewer[27276]: Child: 37832: CProtoV4Down: processing security types message
<15> 2026-07-29T08:40:33.510Z **** vncviewer[27276]: Child: 37832: CProtoV4Down: Server offers security type Ard(30)
<15> 2026-07-29T08:40:33.510Z **** vncviewer[27276]: Child: 37832: CProtoV4Down: Server offers security type [unknown secType 33](33)
<15> 2026-07-29T08:40:33.510Z **** vncviewer[27276]: Child: 37832: CProtoV4Down: Server offers security type [unknown secType 36](36)
<15> 2026-07-29T08:40:33.510Z **** vncviewer[27276]: Child: 37832: CProtoV4Down: Server offers security type [unknown secType 35](35)
<14> 2026-07-29T08:40:33.510Z **** vncviewer[27276]: Child: 37832: CProtoV4Down: Choosing security type Ard(30)
<15> 2026-07-29T08:40:33.510Z **** vncviewer[27276]: Child: 37832: CProtoV4Down: processing security message
<15> 2026-07-29T08:40:33.663Z **** vncviewer[27276]: Child: 37832: CProtoV4Down: processing security message
<13> 2026-07-29T08:40:33.663Z **** vncviewer[27276]: Child: 37832: CConnection: close: [ProtocolError] Protocol error: key length too large

0

Comments

1 comment
  • Official comment

    Hi Ben,

    Thanks for providing the detailed information and debug logs.

    macOS Tahoe 26 is currently a supported platform for RealVNC Connect, so we’d like to take a closer look at the “Protocol error: key length too large” message you’re seeing.

    Could you please confirm the versions of RealVNC Viewer and RealVNC Server you are using?

    If you’ve already raised a Support ticket for this issue, please also share the ticket number so we can link this Community report with the existing case.

    If not, we may recommend raising a Support ticket so the relevant logs and environment details can be reviewed further.

    Thanks again for documenting the issue so thoroughly.

    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post