PM2: AWS IoT Setup CDK
IoT For SAP CDK Solution
➡️ Background
-
In this section of the lab you will construct a demonstration of how an IoT setup can be used to detect the potential equipment failures based on a Temperature anomaly and create an Service notification to trigger the maintenance activities in SAP.
-
This project contains a set of AWS CDK deployable stacks that contains a common set of resources for testing devices with AWS IoT Core. The purpose of this project is to provide a quick pathway for creating AWS resources commonly used for evaluating and gaining visibility into IoT device behavior.
Technical Considerations:
-
For this workshop you can use any IDE of choice also on your local machine e.g. Visual Code.
-
Install CDK in your local environment. Follow the steps outlined in lab preparation section.
-
You will validate the following services in AWS account after CDK deployment is complete.
Resources created in these CDK stacks include:
- **Testing the temperature anomaly **: You will setup an IoT “Thing” in AWS IoT Core, once you have that setup you will run a small program to simulate data being sent to AWS IoT Core and then use the MQTT Test Client to view each MQTT message payload.
This project is designed for use with devices with cryptographic coprocessors such as the ATECC608. If you are using a device with a protected private key or wish to use your own private key, place the CSR in the certs/ directory with the filename <thing_name>.csr.pem where <thing_name> is the X509 Certificate Subject’s CommonName (e.g. certs/my_device_1.csr.pem). If you do not have a private key and CSR you care to use, they will be created for you on when the stack is deployed.
- Install python in your local environment.
UNLESS OTHERWISE NOTED, ENSURE REGION SET TO US-EAST-1 (N. Virginia)
Step 1: Executing CDK to deploy the AWS resources.
- Clone the cdk-iot-analytics to your local folder of your choice.
git clone https://github.com/aws-samples/aws-iot-sap-condition-monitoring-demo.git
- Execute the following command to set up local environment:
python3 -m venv .venv
source .venv/bin/activate
cd cdk-iot-analytics/
pip install -r requirements.txt
- Execute the following command to configure stack variables in cdk.json in the following path.
cd cdk-iot-analytics
- thing_name - “sap-iot-thing” (Any desired name)
- Type - “Compressor” (Any desired name)
- Equipment - <SAP Equipment number from previous lab. example - 299998888>
- FunctLoc - <SAP Functional location from previous lab. example - MUM1-THA-AB-02>
- temperature_min - “12”
- temperature_max - “14”
- sns_alert_email_topic - “TemperatureAlarm”
- alarm_emails - “
” - odpEntitySetName - “<odpEntitySetName from previous step: ex - NOTIF_CREATESet>”
- odpServiceName - “<odpServiceName from previous step example -ZPM_SERVICE_NOTIFICATION_SRV>”
- sapHostName - “Your SAP host name”
- sapPort - “8000” (For this example we will use sap http port maintained in SAP smicm transaction)
- sapUsername - “SAP user name”
- sapPassword - “password”
- Deploy the latest version of cdk.
Ensure your run npm install -g npm from you local terminal to have the latest version of cdk
npm install -g npm
- Execute the following command to Bootstrap CDK to the target account/region.
cdk bootstrap aws://<account>/<region>
- Validate the if you have bootstrapped to your account.
- Execute the following command to deploy the IoT stack with CDK.
cdk deploy iot -O=iot-outputs.json
- Acknowlegde this message: Do you wish to deploy these changes = Y
- Verify the succesful deployment message in your local ide.
- Deploy the SAP Stack.
- For technical and legal reasons, we do not package some dependencies in this repository, so they must be packaged before deploying the SAP stack. Do this with the command, below:
pip install \
requests \
xmltodict \
-t ./cdk_sap_blog/sap/lambda_assets/layer/python/
- Once requests and xmltodict are packaged for the lambda layer, the stack can be deployed with the command, below:
cdk deploy sap
- Acknowlegde this message: Do you wish to deploy these changes = Y
- Verify the successful deployment message in your local IDE.
- Make sure you confirm the subscription to receive the Alarm notification. Check the email account provided in cdk.json to confirm the SNS notification.
- Deploy the Analytics Stack Integrated with AWS IoT Core—AWS IoT Analytics is fully integrated with AWS IoT Core so it can receive messages from connected devices as they stream in and identify anomaly based the defined ranges for an equipment in Dynamo DB
cdk deploy analytics -O=analytics-outputs.json
- Acknowlegde this message: Do you wish to deploy these changes = Y
- Verify the successful deployment message in your local IDE.
- Login into AWS Cloud Formation to validate the stack for successful deployment.
Step 2: Validate the AWS IoT Resources created as part of the stack.
-
IoT Core : Log into your AWS console and make sure you can access the AWS IoT dashboard by selecting AWS IoT Core in AWS Services search window. Click Manage ->Things->Click on your thing name to validate the Thing attributes.
- X509 Certificate signing request certificate issued and installed on Cloud or IDE when you issue CDK deploy. See the IoT core security section and you local IDE with new folder called cert generated post the CDK deploy command. MQTT message is authenticated based on the x509 certs.
- Simulated data is sent to AWS IoT Core using MQTT Test Client to view each MQTT message payload.
- Equipment type (Compressor) is maintained IoT Core Registry.Which is passed to Dynamo DB to extract the SAP material information.
- IoT Action: AWS IoT rules send data from your simulator to other AWS services. For this lab the payload is parsed and sent to IoT analytics. IoT rules listen for specific MQTT messages, format the data in the message payloads, and send the result to other AWS services.
-
From the IoT Core Console, click ‘Act’/Rules and select the rule created as part of this cdk.
Validate the following values in create a rule screen:
- Name: CDKSAP**
- Using SQL version: 2016-03-23
- Rule query statement: Validate the query.
SELECT parse_time("yyyy-MM-dd'T'HH:mm:ssZ", timestamp() ) AS timestamp, topic(2) AS thingname, e.t AS temperature_degC, e.h AS humidity_percent, e.d AS dewpoint_degC, e.i AS heatIndex_degC, l AS location FROM 'trackercdk'
- Validate the Actions Sections. You Should see an action configured to send message to IoT Analytics.
- You Should see an action configured to send message to Lambda for error handling.
- Open the the above rule Functions on the Lambda console to validate the code.
- AWS IoT Analytics: Automates the steps required to analyze data from IoT devices. AWS IoT Analytics filters, transforms, and enriches IoT data before storing it in a time-series data store for analysis
-
On the AWS IoT Analytics console landing page, Click Channels->To view your channel and validate:
- Channel ID = CDK Channel
- Choose storage type = AWS managed S3 bucket
- Retain data in this store = Based on time
- For = 1 day
Validate the Pipeline: Pipeline activities enrich or transform message attributes, or filter entire messages out of your pipeline. Chaining activities together enables you to process and prepare messages before storing them.
- Click edit on Activties and Data Store to explore
Validate the three Activties.
-
Note the Lambda referenced by the IoT Analytics is created as part of the the CDK to get the product range from the Dynamo DB to identify the temperature range and determine if there is a temperature fluctuation in the data sent from the IoT simulator.
-
Open the Functions page on the Lambda console. To verify the code.
-
In addtion Dynamo DB, contain the SAP attributes mapped against the device name (Compressor). When an anomaly is detected the SAP product information is pulled to create a service notification. Login toDynamoDB console to validate the mapping.
Validate the Datastore: On the AWS IoT Analytics console landing page, Click Channels->To view your channel and validate:
-
Data store receives and stores your messages. It is not a database but a scalable and queryable repository of your messages. Processed data store messages are stored in an Amazon S3 bucket managed by AWS IoT Analytics or in one managed by you.
-
Validate the SQL query. This query selects the most recent location information, but averages the temperature reading over the period. Also note that the range from Dynamo was previously injected into the message and the most recent value is selected here.
SELECT max(timestamp) AS endTime, (to_unixtime(from_iso8601_timestamp(max(timestamp))) - to_unixtime(from_iso8601_timestamp(min(timestamp)))) AS durS, max_by(thingname, timestamp) as thingname, avg(temperature_degC) AS temperature_degC_mean, avg(humidity_percent) AS humidity_percent_mean, max_by(range.temperature.max, timestamp) AS maxTemp_degC, max_by(range.temperature.min, timestamp) AS minTemp_degC, max_by(location, timestamp) AS location, max_by(registry.attributes.Equipment, timestamp) AS equipment, max_by(registry.attributes.FunctLoc, timestamp) AS functloc, max_by(registry.attributes.Type, timestamp) AS type FROM trackerdatastore
- This lab uses the AWS IoT Data store output as a ‘Service-managed store’ with 1 day data retention.
- Setup a Data store output as a ‘Service-managed store’ with 1 day data retention. A data store receives and stores your messages. It is not a database but a scalable and queryable repository of your messages. Processed data store messages are stored in an Amazon S3 bucket managed by AWS IoT Analytics or in one managed by you.
NOTE: This rule inserts an ISO8601 timestamp, the thingname (extracted from the topic), and expands the “shorthand” references to environmental conditions into more descriptive property names.
- AWS IoT Analytics Data sets: From the IoT Analytics Console, Click on ‘Data sets’ in the IoT Analytics Console and validate the data set. A data store receives and stores your messages. It is not a database but a scalable and queryable repository of your messages. Processed data store messages are stored in an Amazon S3 bucket managed by AWS IoT Analytics or in one managed by you. Validate!
- Validate the IoT Events: From the IoT Events Console. Click the hamburger menu->Inputs.
- AWS IoT Events enables you to monitor your equipment or device fleets for failures or changes in operation, and to trigger actions when such events occur. AWS IoT Events continuously watches IoT sensor data from devices, processes, applications, and other AWS services to identify significant events so you can take action.
- Validate Lambda for SAP integration. Open the Functions page on the Lambda console.
- Open the Lambda function CDK-SAP-Lab-OData-Function and validate the code and environment variables that connects SAP.
- IoT Events - Detector models Login to your IoT Events Console, create a new Detector Model (hamburger menu/Detector models/Create detector model). Select the Detector Model and export the Json to review the code. You should see code written to listen for any anomaly and trigger the Lambda to create a SAP notification for maintenace activities.
Json code:
{
"detectorModelDefinition": {
"states": [
{
"stateName": "OverTemp",
"onInput": {
"events": [],
"transitionEvents": [
{
"eventName": "overTempTooLong",
"condition": "timeout(\"hiTempDur\")",
"actions": [],
"nextState": "OverTempAlarm"
},
{
"eventName": "tempBelowMax",
"condition": "convert(Decimal,$input.CDKSAPBlogEventsInput.temperature_degC_mean) < convert(Decimal,$input.CDKSAPBlogEventsInput.maxTemp_degC)",
"actions": [],
"nextState": "TempInRange"
}
]
},
"onEnter": {
"events": [
{
"eventName": "startTimer",
"condition": "true",
"actions": [
{
"setTimer": {
"timerName": "hiTempDur",
"seconds": 300,
"durationExpression": null
}
}
]
}
]
},
"onExit": {
"events": [
{
"eventName": "deleteTimer",
"condition": "true",
"actions": [
{
"clearTimer": {
"timerName": "hiTempDur"
}
}
]
}
]
}
},
{
"stateName": "TempInRange",
"onInput": {
"events": [
{
"eventName": "readTemp",
"condition": "true",
"actions": [
{
"setVariable": {
"variableName": "temperature_degC",
"value": "convert(Decimal, $input.CDKSAPBlogEventsInput.temperature_degC_mean)"
}
}
]
}
],
"transitionEvents": [
{
"eventName": "tempAboveMax",
"condition": "convert(Decimal, $input.CDKSAPBlogEventsInput.temperature_degC_mean) > convert(Decimal, $input.CDKSAPBlogEventsInput.maxTemp_degC)",
"actions": [],
"nextState": "OverTemp"
},
{
"eventName": "tempBelowMin",
"condition": "convert(Decimal,$input.CDKSAPBlogEventsInput.temperature_degC_mean) < convert(Decimal,$input.CDKSAPBlogEventsInput.minTemp_degC)",
"actions": [],
"nextState": "UnderTemp"
}
]
},
"onEnter": {
"events": [
{
"eventName": "inRange",
"condition": "true",
"actions": [
{
"lambda": {
"functionArn": "arn:aws:lambda:us-east-1:<aws-account number>:function:CDK-SAP-Lab-OData-Function",
"payload": {
"contentExpression": "'{\n \"d\": {\n \"Equipment\": \"${$input.CDKSAPBlogEventsInput.Equipment}\",\n \"FunctLoc\": \"${$input.CDKSAPBlogEventsInput.FunctLoc}\",\n \"ShortText\": \"Temperature Alarm[TempInRange:inRange]\",\n \"LongText\": \"Temperature Alarm in ${$input.CDKSAPBlogEventsInput.Type} Motor\"\n }\n}'",
"type": "JSON"
}
}
}
]
},
{
"eventName": "sendAlert",
"condition": "true",
"actions": [
{
"lambda": {
"functionArn": "arn:aws:lambda:us-east-1:<aws-account number>:function:CDK-SAP-Lab-OData-Function",
"payload": {
"contentExpression": "'{\n \"d\": {\n \"Equipment\": \"${$input.CDKSAPBlogEventsInput.Equipment}\",\n \"FunctLoc\": \"${$input.CDKSAPBlogEventsInput.FunctLoc}\",\n \"ShortText\": \"Temperature Alarm[TempInRange.sendAlert]\",\n \"LongText\": \"Temperature Alarm in ${$input.CDKSAPBlogEventsInput.Type} Motor\"\n }\n}'",
"type": "JSON"
}
}
}
]
}
]
},
"onExit": {
"events": []
}
},
{
"stateName": "UnderTemp",
"onInput": {
"events": [],
"transitionEvents": [
{
"eventName": "underTempTooLong",
"condition": "timeout(\"loTempDur\")",
"actions": [],
"nextState": "UnderTempAlarm"
},
{
"eventName": "tempAboveMin",
"condition": "convert(Decimal,$input.CDKSAPBlogEventsInput.temperature_degC_mean) > convert(Decimal,$input.CDKSAPBlogEventsInput.minTemp_degC)",
"actions": [],
"nextState": "TempInRange"
}
]
},
"onEnter": {
"events": [
{
"eventName": "startTimer",
"condition": "true",
"actions": [
{
"setTimer": {
"timerName": "loTempDur",
"seconds": 300,
"durationExpression": null
}
}
]
}
]
},
"onExit": {
"events": [
{
"eventName": "deleteTimer",
"condition": null,
"actions": [
{
"clearTimer": {
"timerName": "loTempDur"
}
}
]
}
]
}
},
{
"stateName": "OverTempAlarm",
"onInput": {
"events": [],
"transitionEvents": [
{
"eventName": "tempBelowMax",
"condition": "convert(Decimal,$input.CDKSAPBlogEventsInput.temperature_degC_mean) < convert(Decimal,$input.CDKSAPBlogEventsInput.maxTemp_degC)",
"actions": [],
"nextState": "TempInRange"
}
]
},
"onEnter": {
"events": [
{
"eventName": "sendAlert",
"condition": "true",
"actions": [
{
"lambda": {
"functionArn": "arn:aws:lambda:us-east-1:<aws-account number>:function:CDK-SAP-Lab-OData-Function",
"payload": {
"contentExpression": "'{\n \"d\": {\n \"Equipment\": \"${$input.CDKSAPBlogEventsInput.Equipment}\",\n \"FunctLoc\": \"${$input.CDKSAPBlogEventsInput.FunctLoc}\",\n \"ShortText\": \"Temperature Alarm[OverTempAlarm.sendAlert]\",\n \"LongText\": \"Temperature Alarm in ${$input.CDKSAPBlogEventsInput.Type} Motor\"\n }\n}'",
"type": "JSON"
}
}
}
]
}
]
},
"onExit": {
"events": []
}
},
{
"stateName": "UnderTempAlarm",
"onInput": {
"events": [],
"transitionEvents": [
{
"eventName": "tempAboveMin",
"condition": "convert(Decimal,$input.CDKSAPBlogEventsInput.temperature_degC_mean) > convert(Decimal,$input.CDKSAPBlogEventsInput.minTemp_degC)",
"actions": [],
"nextState": "TempInRange"
}
]
},
"onEnter": {
"events": [
{
"eventName": "sendAlert",
"condition": "true",
"actions": [
{
"lambda": {
"functionArn": "arn:aws:lambda:us-east-1:<aws-account number>:function:CDK-SAP-Lab-OData-Function",
"payload": {
"contentExpression": "'{\n \"d\": {\n \"Equipment\": \"${$input.CDKSAPBlogEventsInput.Equipment}\",\n \"FunctLoc\": \"${$input.CDKSAPBlogEventsInput.FunctLoc}\",\n \"ShortText\": \"Temperature Alarm[UnderTempAlarm.sendAlert]\",\n \"LongText\": \"Temperature Alarm in ${$input.CDKSAPBlogEventsInput.Type} Motor\"\n }\n}'",
"type": "JSON"
}
}
}
]
}
]
},
"onExit": {
"events": []
}
}
],
"initialStateName": "TempInRange"
},
"detectorModelDescription": "Detect temperature in a given range.",
"detectorModelName": "trackerEventDetectorModel",
"evaluationMethod": "BATCH",
"key": "thingname",
"roleArn": "arn:aws:iam::<aws-account number>:role/trackerEventDetectorRole"
}
-
From the IoT Events Console, click on the Detector Model (hamburger menu/Detector models.
-
Click ‘Import detector model’ and upload the model JSON (trackerEventDetectorModel.json. Validate the events.
- Testing the Anomaly:
-
The simulator uses the temperature_min/temperature_max variables you defined in cdk.json to report temperatures uniformly to be a few degrees hotter than the maximum (see simulator.py:L50).
-
See the minimum temperature and maximum temperature in your Dynamo DB table maintained as part of your CDK deploy.
- On the AWS IoT Analytics console landing page, Click Datasets->Click run now in actions to validate the payload that is sent to IoT Events and Amazon S3 for storage.
- Next, from the IoT Events Console. Click the hamburger menu and click the Event Detector Model –> Click your Detector Model –> Click the key value in Detector section(IoT Inputs) –>Validate the state and temperature value. Here you should the values within Dynamo DB temperature range.
- Modify the temperature values which is outside the range of the temperature sent by the simulator payload. From min 12 and max 14 to any desired values.
- Go back to IoT Events Console. Click the hamburger menu and click the Event Detector Model –> Click your Detector Model –> Click the key value in Detector section(IoT Inputs) –>Validate the state and temperature value. Here you should see overtemp value based on the above screen shot values in Dynamo DB.
- Event Detector will see if the state of overtemp based on the temperature readings from the simulator. If the anomaly continues for > 5 minutes an alarm should be triggered to create a service notification in SAP and sent and SNS notification to the SNS subscriber list.
-
Validate the service notification in SAP. Go to SAP Transaction IW23
- Input the SAP Service Maintenance Notification generated. Click enter and verify the entries.
Congratulations, you have completed the integration between SAP and AWS AWS IoT services to enable Maintenance with AWS CDK.