Test Edge Drivers using SmartThings Test Suite
test edge drivers using smartthings test suite objective learn how to identify and resolve issues when deploying edge drivers using smartthings test suite overview smartthings test suite is a tool for testing iot device integrations within the smartthings platform this solution provides a seamless certification process, allowing developers of smartthings hub connected devices to submit their products for certification without the need for manual testing this accelerates the certification timeline and offers a more cost-effective path to certifying these devices the key feature of this self-testing tool is it contains an automated testing suite that covers critical certification criteria, ranging from functionality to performance tests the tool also provides real-time feedback that gives detailed information on the device's compliance status, allowing for quick identification and resolution of any issues lastly, it has an intuitive and user-friendly interface that ensures a seamless experience for developers of all levels set up your environment notefor sdc24 attendees, skip this step as it’s already done for you you will need the following host pc running on windows 10 or higher or ubuntu 20 04 x64 visual studio code latest version recommended devices connected on the same network android mobile device with smartthings app installed with android 10 or higher smartthings station or smartthings hub onboarded with samsung account smartthings v4 multipurpose sensor or lightify tunable white 60 light bulb notemake sure that your devices are connected to your smartthings app sample code here is a sample code for this code lab download it and start your learning experience! test suite sample code 185 4 kb install smartthings cli you need to install smartthings cli as this is the main tool for developing apps and drivers for smartthings edge drivers to install smartthings cli, open a web browser and download the smartthings msi installer from the latest release notefor other operating systems, download the appropriate zipped binary and install it on your system path open the smartthings cli setup in the downloaded file, then click next accept the license agreement terms, then click next select the destination path for installation and click next to begin the installation process, click install notethe windows installer may display a warning titled windows protected your pc to continue the installation, click more info > run anyway complete the setup by clicking finish to verify if smartthings cli is installed correctly, open the command prompt and run this command smartthings --version view and run available commands for smartthings cli with this command smartthings --help for a full list of commands, visit the smartthings cli commands notethe smartthings cli supports an automatic login flow that launches a browser window, prompting the user to log in with samsung account and grant the cli permissions to access the user's account start your project after downloading and extracting the sample code containing the project files, click file > open folder in visual studio code to open it locate the sample code file directory and click select folder once finished, the project files are seen on the explorer menu configure custom edge drivers open your command prompt or terminal and follow the corresponding instructions depending on your device availability make sure that the path directory in your cli contains the project file smartthings multipurpose sensor in the terminal, type the following command to build and upload your edge driver package to the smartthings cloud smartthings edge drivers package drivers/codelab-zigbee-contact create a new channel for your edge driver and enter the following channel details smartthings edge channels create channel name smartthings test suite demo channel description channel for sdc2024 channel terms of service url www smartthings com enroll your hub in your newly created channel and select the corresponding channel and hub smartthings edge channels enroll assign your driver to the created channel smartthings edge channels assign install the created edge driver from your channel to your own hub smartthings edge drivers install confirm that the correct version of the driver is present in your hub smartthings edge drivers installed select the edge driver for this device smartthings edge drivers switch lightify tunable white 60 bulb in the terminal, type the following command to build and upload your edge driver package to the smartthings cloud smartthings edge drivers package drivers/codelab-zigbee-switch create a new channel for your edge driver and enter the following channel details smartthings edge channels create channel name smartthings test suite demo channel description channel for sdc2024 channel terms of service url www smartthings com enroll your hub in your newly created channel and select the corresponding channel and hub smartthings edge channels enroll assign your driver to the created channel smartthings edge channels assign install the created edge driver from your channel to your own hub smartthings edge drivers install confirm that the correct version of the driver is present in your hub smartthings edge drivers installed select the edge driver for this device smartthings edge drivers switch test your device on your web browser, go to smartthings test suite, login to your samsung account and follow the corresponding instructions depending on your device availability smartthings multipurpose sensor on the test suite, look for your device, click menu icon > prepare new test under the compatible capabilities, select all capabilities except for battery, and click start during the test execution, perform the indicated user actions for every test case if there are any it might cause incorrect test results if user actions are not performed tipyou may view the real-time sensor states of the device in the smartthings mobile app view the test summary after the test it returns a failed test that you are going to resolve in the next step lightify tunable 60 white bulb on the test suite, look for your device, click menu icon > prepare new test under the compatible capabilities, select all capabilities after you've selected the capabilities, click start warningduring the test execution, observe the behavior of the bulb it might cause incorrect test results if automated tests are interrupted view the test summary after the test it returns a failed test that you are going to resolve in the next step resolve test failures smartthings multipurpose sensor the test logs contain basic information about the test results and specific test cases, providing technical context to users for efficient troubleshooting download the test logs by navigating to the bottom page of the test summary > show full test details > download log to understand the test logs, its structure follows this schema {execution timestamp} device node path node type [node state] {execution message} in the downloaded test log, two test cases failed with the following error [failed] initialize states following states were not set correctly [contact any other state than "open" on main contactsensor] [failed] send command and validate some events didn't happen [contact "closed" on main contactsensor] some states aren't final [contact "closed" on main contactsensor] in the first error log, it appears that the test suite cannot change the device's state to anything other than an open state in the second error log, the test suite tries to change its state to closed, but to no avail it is confirmed in the capability definition that the contact sensor has only two states open and closed therefore, the device is constantly in an open state and unable to change to a closed state with these information, you can start troubleshooting by going to drivers > codelab-zigbee-contact > multi-sensor > init lua and look for incorrect code implementation with these keywords open, closed, contactsensor it can be seen in the zone_status_change_handler and zone_status_handler functions that there are code blocks on comment this might be a result of someone developing this code have changed this part of code for debugging and forgot to uncomment this part uncomment this code block from zone_status_change_handler function if not device preferences["certifiedpreferences garagesensor"] then contactsensor_defaults ias_zone_status_change_handler driver, device, zb_rx end uncomment this code block from zone_status_handler function if not device preferences["certifiedpreferences garagesensor"] then contactsensor_defaults ias_zone_status_attr_handler driver, device, zone_status, zb_rx end remove this line of code from zone_status_change_handler and zone_status_handler functions device emit_event_for_endpoint zb_rx address_header src_endpoint_value, capabilities contactsensor contact open save the file and update the driver by invoking the same cli commands that were also used during the configuration of custom edge drivers smartthings edge drivers package drivers/codelab-zigbee-contact smartthings edge channels assign smartthings edge drivers install again, go to smartthings test suite, select your device, click menu > prepare new test ensure that all compatible capabilities are selected, with the exception for battery again, start the test and perform indicated user actions for every test case if there are any now, all tests are passed! lightify tunable white 60 bulb the test logs contain basic information about the test results and specific test cases, providing technical context to users for efficient troubleshooting download the test logs by navigating to the bottom page of the test summary > show full test details > download log to understand the test logs, its structure follows this schema {execution timestamp} device node path node type [node state] {execution message} in the downloaded test log, one test case failed with the following error [failed] send command and validate some events didn't happen [colortemperature maximum 7500k on main colortemperature] some states aren't final [colortemperature maximum 7500k on main colortemperature] in the error log, it directs to an issue for setting an incorrect maximum colortemperature value the configuration on your edge driver is set to 7500k you can start to troubleshoot by looking for the bulb's color temperature rating either from the device packaging or the device manufacturer website with these information, you can start troubleshooting by going to drivers > codelab-zigbee-switch > profiles > rgbw-bulb yml and look for lines that declares the colortemperature value change the colortemperature range declaration range [ 2700, 6500 ] save the file and update the driver by invoking the same cli commands that were also used during the configuration of custom edge drivers smartthings edge drivers package drivers/codelab-zigbee-switch smartthings edge channels assign smartthings edge drivers install again, go to smartthings test suite, select your device, click menu > prepare new test under the compatible capabilities, select all capabilities again, start the test and perform indicated user actions for every test case if there are any now, all tests are passed! you're done! congratulations! you have successfully achieved the goal of this code lab now, you can test your edge driver for smartthings devices using smartthings test suite! to learn more about smartthings test suite, visit smartthings edge architecture smartthings developer console