SAML Authentication failing after upgrade from 4.2.x to 4.3+

After upgrading Arcadia to 4.3 and beyond (i.e. 4.4.x) from previous versions such as 4.2, you may hit a SAML Signature error like this after restarting:

2018-12-04 09:32:09,367 INFO P1 6015 CP Server Thread-8 Invalid or malformed SAML Assertion.
Traceback (most recent call last):
  File "/opt/arcadia/lib/venv/lib64/python3.6/site-packages/djangosaml2-0.17.2.1-py3.6.egg/djangosaml2/views.py", line 273, in assertion_consumer_service
    response = client.parse_authn_request_response(xmlstr, BINDING_HTTP_POST, outstanding_queries)
  File "/opt/arcadia/lib/venv/lib64/python3.6/site-packages/pysaml2-4.5.0.1-py3.6.egg/saml2/client_base.py", line 702, in parse_authn_request_response
    binding, **kwargs)
  File "/opt/arcadia/lib/venv/lib64/python3.6/site-packages/pysaml2-4.5.0.1-py3.6.egg/saml2/entity.py", line 1138, in _parse_response
    response = response.loads(xmlstr, False, origxml=origxml)
  File "/opt/arcadia/lib/venv/lib64/python3.6/site-packages/pysaml2-4.5.0.1-py3.6.egg/saml2/response.py", line 512, in loads
    self._loads(xmldata, decode, origxml)
  File "/opt/arcadia/lib/venv/lib64/python3.6/site-packages/pysaml2-4.5.0.1-py3.6.egg/saml2/response.py", line 337, in _loads
    **args)
  File "/opt/arcadia/lib/venv/lib64/python3.6/site-packages/pysaml2-4.5.0.1-py3.6.egg/saml2/sigver.py", line 1750, in correctly_signed_response
    raise SignatureError("Signature missing for response")
saml2.sigver.SignatureError: Signature missing for response

To fix the issue, you should add this line in your custom SAML authentication configuration, which is stored in the Arcadia Visualization Server Safety Valve (settings_cm.py) in Cloudera Manager, or Arcviz Settings in Ambari:

'want_response_signed': False

You should add this line in the section of the SAML configuration that starts like this:

SAML_CONFIG = {
  # full path to the xmlsec1 binary program
  'xmlsec_binary': '/usr/bin/xmlsec1',
 
  # your entity id, usually your subdomain plus the url to the metadata view
  'entityid': 'my_arcadia_application',
  # directory with attribute mapping
  'attribute_map_dir': '/var/lib/arcadia/saml/attribute-maps',
 
  # this block states what services we provide
  'service': {
    # we are just a lonely SP
    'sp': {
      'name': 'Federated Arcadia Data SP',
      'name_id_format': saml.NAMEID_FORMAT_UNSPECIFIED1,
      'allow_unsolicited': True,
      'logout_requests_signed': False,
      'authn_requests_signed':False,
      'want_response_signed':False,  # Add new line here!
     ........
1 Like

A few more notes that might help debugging.

MAPPING IDP SETTING TO ARCVIZ SETTING

#1 keyclock "Sign Documents"
'want_response_signed': False, 

#2 keyclock "Sign Assertions" (Unnecessary if Sign Documents is enabled) 
'want_assertions_signed': True,

#3 keycloak "Client Signature Required" 
'authn_requests_signed': True,`

You can skip #2 once you have #1.

#1 & #2 pysaml configs govern our requirement that things we receive from the IDP must be signed. They are allowed to be signed even if both #1 & #2 configs are set to False.
However, xmlsec1 is always used to validate when a signature is received in an xml saml doc. Any failure to validate the signature will fail the login saml2.sigver.SignatureError: Failed to verify signature . This will happen regardless of the “want_…*” configs.

MISMATCH BETWEEN IDP AND ARCVIZ SETTING

If you turn OFF “sign documents” on the IDP side (screenshot below) and in arcviz you have ‘want_response_signed’: True, then there’s a mismatch and arcviz will throw signature validation error similar to the below. Arcviz will start complaining saying it’s trying to validate signature and not finding anything.

This is the error shown in the top thread above.

2018-10-03 13:57:41,535 ERROR P1 90347 Thread-12 Signature Error: Signature missing for response
2018-10-03 13:57:41,538 ERROR P1 90347 Thread-12 XML parse error: Signature missing for response
2018-10-03 13:57:41,539 INFO P1 90347 Thread-12 Invalid or malformed SAML Assertion.
Traceback (most recent call last):
  File "/Users/alex/.pyenv/versions/arcviz_3.6.2/lib/python3.6/site-packages/djangosaml2/views.py", line 271, in assertion_consumer_service
    response = client.parse_authn_request_response(xmlstr, BINDING_HTTP_POST, outstanding_queries)
  File "/Users/alex/.pyenv/versions/arcviz_3.6.2/lib/python3.6/site-packages/saml2/client_base.py", line 702, in parse_authn_request_response
    binding, **kwargs)
  File "/Users/alex/.pyenv/versions/arcviz_3.6.2/lib/python3.6/site-packages/saml2/entity.py", line 1138, in _parse_response
    response = response.loads(xmlstr, False, origxml=origxml)
  File "/Users/alex/.pyenv/versions/arcviz_3.6.2/lib/python3.6/site-packages/saml2/response.py", line 512, in loads
    self._loads(xmldata, decode, origxml)
  File "/Users/alex/.pyenv/versions/arcviz_3.6.2/lib/python3.6/site-packages/saml2/response.py", line 337, in _loads
    **args)
  File "/Users/alex/.pyenv/versions/arcviz_3.6.2/lib/python3.6/site-packages/saml2/sigver.py", line 1750, in correctly_signed_response
    raise SignatureError("Signature missing for response")
saml2.sigver.SignatureError: Signature missing for response

SIGNATURE VALIDATION LOGS

When we go through the signature validation check path, we get these two messages in the logs here:

  1. one which prints out the certs needed to verify signature
  2. second the xmlsec procedure to verify the signature that comes in

2018-10-04 10:31:39,781 DEBUG P1 94775 Thread-55 ==== Certs from metadata ==== None: [(<tempfile._TemporaryFileWrapper object at 0x11291d6a0>, '/var/folders/qg/tg7h4whd4s56jxfjc7vfz8lc0000gn/T/tmpr6swqbig.pem')] ==== ... 2018-10-04 10:32:39,292 DEBUG P1 94775 Thread-55 xmlsec command: /usr/local/bin/xmlsec1 --verify --pubkey-cert-pem /var/folders/qg/tg7h4whd4s56jxfjc7vfz8lc0000gn/T/tmpr6swqbig.pem --id-attr:ID urn:oasis:names:tc:SAML:2.0:protocol:Response --store-signatures --node-id ID_3433c83c-cb82-4de6-ba44-e5eccb894929 --output /var/folders/qg/tg7h4whd4s56jxfjc7vfz8lc0000gn/T/tmprv6em5uo.xml /var/folders/qg/tg7h4whd4s56jxfjc7vfz8lc0000gn/T/tmp5qsodiwq.xml

That xmlsec1 --verify is what is run to check the signature. Explicitly run as part of the validate_signature process. This goes into how xmlsec1 works. Saml module on the arcviz side gets a stdout/stderr response from the xmlsec1 call to determine if an error occurred. Nothing suggests that it does anything with the disk written files.

Assumig you have “sign documents” turned ON on the IDP Side, and arcviz settings ‘want_response_signed’= True, then if xmlsec outputs fail or error, it will stop login.

1 Like