Quality Engineering & Testing for the Internet of Things (IoT) – A brief overview!!

Let’s explore the world of new opportunity in the area of Quality Engineering.

With the significant increase of IOT deployments and millions of connected devices in the IOT world, testing is still an area which is falling behind. It’s time we should focus on developing a robust Quality Engineering framework for the connected world.

The time when there is a significant increase in IOT deployments, testing is still an area which is falling behind. There are more connected devices on the planet today than the total human population, and this number will continue to grow manifold. IoT testing teams are facing an unprecedented challenge today. How could they prove the reliability of their IoT infrastructure to handle millions of connected devices? How would they ensure the coverage of the entire spectrum of possible real world IoT use case scenarios? How will they test the usability & performance scenarios with millions of devices sending data together?

Applying quality engineering processes and strategies in the area of IOT is much more complex than the normal software applications.

IOT overview & Architecture

IOT is an amalgamation of technologies. Unlike human end-users in case of common application testing, in the IOT world the initiation of the functional flow mostly starts from a machine / physical device or a sensor connected through short range wireless protocols to the IOT gateway which is further connected to the cloud interface using long-range wireless technologies. The collected data is further used by IOT platforms, stored in the databases and other systems where device management, device configurations, business logic, Data analysis, actuation and decision making takes place along with other required services.

IOT implementation happens across different functional domains ranging from Retail, logistics, mining, agriculture, irrigation, healthcare, infrastructure development, security and a wide range of industries. Hence the solutions and business cases differ significantly.

IOT end to end system covers a wide range of protocols across all SEVEN OSI layers. Hence the tester should have a proper understanding of sensors, networking and communication technologies along with an overall understanding of Application systems, Cloud infrastructure, and a basic knowledge of AI and ML is needed for any Quality Engineering expert for defining an end to end test strategy for any IOT project.

Testing in the world IOT: -

The traditional QA approach and strategies will not suffice the testing of IOT applications. A combination of varying sensor data, device states, combined with possible error scenarios, complex device to device interaction and unreliable network conditions will generate nearly an infinite amount of test vectors. The testing of IOT technology & infrastructure has become a big challenge because of

1. Too many areas to test – A wide range of sensors, nodes, gateways, communications and technology areas

2. Hardware & Software mesh - closely coupled among various hardware and software components

3. Heterogenous interfaces & lack of interoperability– Multi vendor multi-protocol devices, difficult to integrate with each other because of different policies and SLA’s

4. Complexity – Functionality & field Conditions. Diversification in IoT implementation areas and their corresponding functional domains. The number of IOt devices ranges from 100’s to millions.

5. Security vulnerabilities – Data security and conformance primarily between the sensors and the nodes.

The need of the hour: -

The test team should be well versed in various technology areas. They should upgrade themselves continuously to cater different customer needs. The testing team should have people with

1. Extensive knowledge of IOT technologies

2. Domain Knowledge (For different industry Domain areas)

3. Knowledge of Networking and communication Protocols

4. Knowledge about Sensors & Actuators

5. Knowledge about Data Science & Analytics

6. Knowledge about Databases (e.g. - NoSQL databases like MongoDB)

7. Understanding of different application & messaging protocols (MQTT, CoApp).

8. Basic knowledge of prominent IOT platforms (ThingWorx, AWS IOT, IBM Watson etc)

9. Understanding of AI and ML is surely an advantage

IOT Testing Framework

The primary consideration for any IOT testing is to have a detailed understanding of the use case of the application. The test approach should primarily be distributed across the four layers in two different test segments.

Segment 1 : For the Devices, gateway & the connectivity Layer

Segment 2 : IOT Application & Service Layer

Types of Testing for IOT eco Systems: -

IOT applications also follows various testing types from our regular application testing life cycle but certain test types have very high significance as far as IOT application ecosystem is concerned. The IOT test strategy should extensively focus on

1. Security testing

2. Performance Testing

3. Interoperability / Compatibility Testing

4. Testing on the POC level

5. Device Connectivity

6. Data quality and reliability

7. Protocol testing

8. Device power / Backup / Battery Life /

9. Simulation and Replication

10. Testing the Cost of implementation

11. Network performance / Reliability & throughput

12. Events / Triggers / Alarms / Actuations / Actions

13. Security

14. Scalability & Usability

15. Business logic

In normal application Functional testing majority of the above areas are not tested so extensively. The above areas are tested in addition or combination with Functional, Integration & regression testing.

I will talk about the above-mentioned testing areas in detail in my future articles. I am also planning to initiate a training program for IOT testing for experienced testing professionals in six weeks’ time. In case you are interested please connect with me through LinkedIn.