Python과 AWS IoT 연동

예전에 AWS IoT를 연동한 Swift 프로그래밍을 할 때에는 AWS IoT 세팅을 직접 한 것이 아니었기에, 이번에 Python 서버와 연동하는 부분을 구현하려고 하니 AWS IoT 세팅에서 시행착오를 겪게 되었다.

Python으로 AWS IoT를 연동하려면 AWSIoTPythonSDK를 설치하여 사용하면 된다.

https://github.com/aws/aws-iot-device-sdk-python

하지만, AWS S3와 달리 AWS IoT 연동을 위해서는 좀더 복잡한 인증 과정이 필요하다.

이 때문에 한참을 삽질하다가, 내가 이전에 포스팅한 글의 내용에서 힌트를 얻었다.

* 주의할 점은, 샘플 소스의 경우 인증서를 사용하여 MQTT 브로커에 접속하도록 되어 있는데 AWS IoT를 비인증 접속하여 사용할 수 있도록 했다면 connectUsingWebSocket 메소드로 접속을 하도록 변경하면 된다.

iotDataManager.connect( withClientId: uuid, cleanSession:true, certificateId:myImages[0], statusCallback: mqttEventCallback)

iotDataManager.connectUsingWebSocket(withClientId: uuid, cleanSession: true, statusCallback: mqttEventCallback)

"비인증 접속"이라니... 혹시나 해서 검색해보니 다음과 같은 포스팅이 나온다.

http://webframeworks.kr/tutorials/weplanet/17IOT-Policy/

웹소켓 접속을 하기 위해서는 프론트엔드에 IOT에서 설정한 인증서를 설치하거나, Cognito Identity Pool로 인증된 유저에 Policy를 연동시키는 방법이 있다. Cognito에 로그인 없이 비인증된 권한으로도 Cognito와 연결할 수 있지만, 우리는 Cognito를 사용하기 때문에 Identity Pool에 권한을 연동하는 방법을 사용한다.

* 여기서 중요한 것은 "User Pools"는 필요없고, 새로운 Identity Pool을 하나 추가하고, Unauthenticated role과 Authenticated role에 지정되어 있는 Role에 아래의 JSON을 넣어주고 IAM - Role 페이지로 가서 앞의 2개의 Role에 각각 IoT 관련 10개의 Policies를 추가해주기만 하면 된다.

게다가 이전에 사용하던 Swift용 AWS IoT 연동 예제의 README.md 파일에 이를 위한 Amazon Congnito Console 세팅 방법이 자세히 나온다.

https://github.com/awslabs/aws-sdk-ios-samples/tree/master/IoT-Sample/Swift

1. This sample requires Cognito to authorize to AWS IoT in order to create a device certificate. Use Amazon Cognito to create a new identity pool:

   1. In the Amazon Cognito Console, press the Manage Federated Identities button and on the resulting page press the Create new identity pool button.

   2. Give your identity pool a name and ensure that Enable access to unauthenticated identities under the Unauthenticated identities section is checked. This allows the sample application to assume the unauthenticated role associated with this identity pool. Press the Create Pool button to create your identity pool.

      Important: see note below on unauthenticated user access.

   3. As part of creating the identity pool, Cognito will setup two roles in Identity and Access Management (IAM). These will be named something similar to: Cognito_PoolNameAuth_Role and Cognito_PoolNameUnauth_Role. You can view them by pressing the View Details button. Now press the Allow button to create the roles.

   4. Save the Identity pool ID value that shows up in red in the "Getting started with Amazon Cognito" page, it should look similar to: `us-east-1:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" and note the region that is being used. These will be used in the application code later.

   5. Now we will attach a policy to the unauthenticated role which has permissions to access the required AWS IoT APIs. This is done by first creating an IAM Policy in the IAM Console and then attaching it to the unauthenticated role. Search for "pubsub" and click on the link for the unauth role. Click on the "Add inline policy" button and add the following example policy which can be used with the sample application. This policy allows the application to create a new certificate (including private key) as well as attach an existing policy to a certificate.

      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Action": [
              "iot:AttachPrincipalPolicy",
              "iot:CreateKeysAndCertificate",
              "iot:CreateCertificateFromCsr"
            ],
            "Resource": [
              "*"
            ]
          }
        ]
      }
      

      More information on AWS IAM roles and policies can be found here. More information on AWS IoT policies can be found here.

      Note: to keep this example simple it makes use of unauthenticated users in the identity pool. This can be used for getting started and prototypes but unauthenticated users should typically only be given read-only permissions if used in production applications. More information on Cognito identity pools including the Cognito developer guide can be found here.

2. Note that the application does not actually create the AWS IoT policy itself, rather it relies on a policy to already be created in AWS IoT and then makes a call to attach that policy to the newly created certificate. To create a policy in AWS IoT,

   1. Navigate to the AWS IoT Console and press the Get Started button. On the resulting page click on Secure on the side panel and the click on Policies.

   2. Click on Create a Policy

   3. Give the policy a name. Note this name as this is the string you will use in the application when making the attach policy API call.

   4. The policy should be created to allow connecting to AWS IoT as well as allowing publishing, subscribing and receiving messages on whatever topics you will use in the sample application. Below is an example policy. This policy allows access to all topics under your AWS IoT account. To scope this policy down to specific topics specify them explicitly as ARNs in the resource section: "Resource": "arn:aws:iot:<REGION>:<ACCOUNT ID>:topic/mytopic/mysubtopic". Note that the first topic is an ARN specifer so this example actually specifies the topic mytopic/mysubtopic.

   5. To add this policy, click on Advanced Mode and replace the default policy with the following text and then click the Create button.

      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Action": "iot:Connect",
            "Resource": "*"
          },
          {
            "Effect": "Allow",
            "Action": [
              "iot:Publish",
              "iot:Subscribe",
              "iot:Receive"
            ],
            "Resource": "*"
          }
        ]
      }

이 Swift 연동 예제에서는 p12 파일을 생성해서 사용하도록 되어 있는데, Python 예제에서는 올바른 인증서 파일이 아니라고 에러가 뜬다. 그래서 이것 저것 찾다가 새로운 단서를 얻었다.

https://github.com/aws/aws-iot-device-sdk-python/issues/33

Maybe it will help this thread, if I describe my steps too.

First, based on information in https://aws.amazon.com/blogs/mobile/use-your-own-certificate-with-aws-iot/I generate my keys:

1. openssl genrsa -out sampleCACertificateOne.key 2048
2. openssl req -x509 -new -nodes -key sampleCACertificateOne.key -sha256 -days 365 -out sampleCACertificateOne.pem
3. ... at this point grab the registration code ...
4. openssl genrsa -out privateKeyVerificationOne.key 2048
5. openssl req -new -key privateKeyVerificationOne.key -out privateKeyVerificationOne.csr
6. openssl x509 -req -in privateKeyVerificationOne.csr -CA sampleCACertificateOne.pem -CAkey sampleCACertificateOne.key -CAcreateserial -out privateKeyVerificationOne.crt -days 365 -sha256
7. ... I insert the registration code to "Common Name" when asked by openssl

Then I go to IoT console, and upload my CS using files sampleCACertificateOne.pem and privateKeyVerificationOne.crt. No problem at this point.

Then I programmatically register a new thing using steps:

1. iot.create_thing()
2. iot.describe_endpoint()
3. iot.create_keys_and_certificates()
4. iot.attach_principal_policy()
5. iot.attach_thing_principal()

And of course, save all keys and credentials for later use. All these steps work 100% fine. I get a new thing, I see it in IoT control panel with its certificates, etc. Everything seems to be just fine.

Then I try to start MQTT client and connect to AWS IoT broker. AWSIoTMQTTClient.configureCredentials() require root CA (I use sampleCACertificateOne.pem), private key and thing certificate (both come in create_keys_and_certificates() call earlier). Connection request fails to:

2017-05-10 10:23:09,063 - AWSIoTPythonSDK.core.protocol.mqttCore - INFO - Connection type: TLSv1.2 Mutual Authentication
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

At this point, it is worth noting that if I download root CA from https://www.symantec.com/content/en/us/enterprise/verisign/roots/VeriSign-Class%203-Public-Primary-Certification-Authority-G5.pem and run the whole process with exact same steps, the SYSTEM WORKS!

Hence, I suspect that this is related to the key generation process itself (maybe wrong params), or a bug in AWS IoT managing the keys.

I hope this helps in any possible way...

혹시나 해서 VeriSign-Class 3-Public-Primary-Certification-Authority-G5.pem 파일을 다운받아서 사용하니 접속이 잘된다!

myAWSIoTMQTTClient.configureCredentials("./cert/VeriSign-Class 3-Public-Primary-Certification-Authority-G5.pem")

문제를 모두 해결하고, 또다시 Python에서 JSON 처리와 Socket.IO 때문에 삽질을 좀더 한 다음에 정상적으로 통신되는 것을 확인하였다.

http://joondong.tistory.com/71

AWS IoT를 이용한 시스템 흐름과 인증에 대해 잘 정리되어 있다.