How we did it: insight into our QA process for Pakistan telehealth initiative

After weeks of hard work, countless meetings, and a successful project delivery later, we’ve decided to pull back the curtains on exactly what it takes to assure the quality of a high-availability healthcare app.

Back in April 2020, when the first wave of the novel Coronavirus was at its peak, VentureDive reached out to the Government of Pakistan to help the country combat the virus through technology. As part of the ‘Digital Pakistan’ initiative, spearheaded by Tania Aidrus, an ex-Google executive, we collaborated with her team to build and launch the COVID-19 telehealth portal. It is a website specially designed to combat the crisis of the pandemic. It allows Pakistani doctors and all healthcare professionals to register on it and volunteer to remotely help the patients who might have COVID-19 symptoms.

Doctors who want to give back during Ramzan: consider volunteering your time by signing up to provide #Covid_19 consultations at https://t.co/6Joc1VFVRM

Deep gratitude to @venturedive, @ptcl, @eocean, @infobip & @ntc for making this happen.@zfrmrza @fslsltn @UdarOfficial

— Tania Aidrus (@taidrus) April 25, 2020

“VentureDive team, I cannot thank you enough on behalf of the entire team for leaning in to help! There has been SO much interest in the platform and what’s interesting is how much interest we are seeing in other verticals. For example, today we had a call with the Law Ministry who are very eager to do something similar to sign up volunteer lawyers to provide free guidance to victims of domestic abuse. I hope this is just the beginning of our working relationship – excited to have started off on doing something that I hope can help thousands of Pakistanis during this time.” — Chief Digital Officer, Digital Pakistan

Healthcare is a very sensitive subject and it was a technology to be used by millions of people across Pakistan. Therefore, it demanded the highest quality, with zero downtime, zero bugs, and intuitive user journeys.  In this blog, we’ve highlighted our experience of testing a portal that was to be used by healthcare professionals to reach out to patients via our technology.

What follows is a tale that tells the challenges we faced during the three-week-long project, and how we resolved them to successfully deliver a web and a mobile application.

The functional, security & scalability challenges of testing the telehealth portal

Before the project kicked off, the quality assurance team at VentureDive gathered the application requirements and shared them with the experts dedicated to working on this. A thorough documentation and sample mockups helped the QA team to begin working on the test plan, test design, and test cases during the development phase. We conducted daily stand-ups so the development & testing teams could stay synced and brainstorm on maneuvering through this project smoothly and in time.  We faced six major challenges during our course:

Time management

The main challenge was racing against time to meet the client’s expectations while ensuring the security of the sensitive healthcare data, and zero glitches within the app. This meant that the QA team had to keep track of every requirement and reporting templates for testing updates that helped the development team fix defects and bugs timely, prior to delivering any milestone to the client.

3rd party integrations

The application was to be integrated with third-party software such as WhatsApp chatbot for doctor-patient communication & telecom operators to enable anonymous calling mechanism and receiving SMS OTP. These were essential for fetching data from official sources and making sure both our applications remained in sync with the whole system we were creating.

Mobile responsiveness

The telehealth portal was supposed to be a hybrid mobile application, which meant that the testing team had to test it across various mobile devices and operating systems to make sure that it was responsive and compatible.

Security

Cyber attacks and threats are a real-world problem today with thousands of networks and websites being compromised each day. To help identify, classify, and address security risks, we performed vulnerability assessment and penetration testing activity including server VA, API penetration testing, and web application penetration testing to identify possible routes an attacker could use to break the system.

System performance

Monitoring the performance of the application was an integral part of building the portal, since we anticipated a large number of users, including doctors and patients. The QA team planned to automate the scripts on JMeter to determine how the system performs in terms of responsiveness and stability under heavy load and a huge volume of data.

Standards

A big challenge for the QA team was to keep the testing practices as standardized as possible even with a short time to spare. VentureDive believes in maintaining the quality of the deliverables as our utmost priority, regardless of the length, complexity, or intent of the project.

Adopting a smart testing strategy for successful project delivery

The process of Alpha testing was done remotely. The QA team collaborated and focused all their efforts towards detect any major defects in data security. We carried out usability, performance and security testing for private and sensitive information in a healthcare setup.

It was pertinent for the QA team to also analyze business criticality, plan around testing efforts in minimum time, make the application usable for thousands of users and ensure that testing was compliant with the Open Web Application Security Project (OWASP) standards.

The test strategy called for having separate environments for development, staging, and production. We performed the following steps in the given order:

Functional testing

Keeping in mind the criticality & nature of health-related projects, requirements had to be precise and the validation had to be perfect. We performed static analysis on requirements followed by actual test execution to meet the requirements and clients’ expectations. Data flow integrity and business rules were repeatedly tested via automated suites in our regression cycles

Tools:

• Postman for API automation & integration testing
• TestRail for test cases and test cycle reporting
• PostgreSQL for data validations
  

Cross browser testing

Browser compatibility was mainly focused on Google Chrome version 80+ on windows. Extended smoke and regression cycle was performed on Firefox and Safari for Windows and Mac respectively.

Tools:

• crossbrowsertesting.com
• Browserstack.com

Responsiveness

Understanding the market trends of portable devices, we analyzed the data of the target audience and performed UI/UX testing on mobile & other portable devices. The application was tested on 6 different Android and iOS devices with different screen sizes and resolutions having different OS versions.

Security testing

Thorough security testing was performed on infrastructure, API, and application level, keeping the top 10 OWASP standards in mind. 

Tools:

• Burpsuite
• ZAP 
• KALI Linux operating system

The QA team identified the following vulnerabilities during the security testing activity:

• Broken access control
• Broken session management
• Disclosure of internal directories
• Unrestricted file upload
• Missing server validation
• Sensitive data exposure
• Brute Force 
• No rate limit
• WAF & ACL implementation

Our goal was to immediately address these issues and recommend further best practices that should be followed as pre-emptive measures against any potential cyber-attacks.

Performance testing

The system undergoing the test was required to have a load-balanced infrastructure supporting thousands of interactions between the patients and the doctors. The flow includes the signup process including uploading of images, populating, and fetching patient data lists and assignments of users one to one. 

Tools:

• Jmeter
• Blazemeter

We analysed all the results, and generated an extensive report using SmartMeter, which was later shared with the stakeholders. The primary issues identified were load balancing, CPU utilisation, and WAF configurations. These were addressed and recommended configurations were made for resolution.

Project delivery

A standard process was put in place to validate the requirements and meet the client’s expectations. After complete and thorough testing, we demonstrated and delivered the project to the client successfully.

Wrap up

Working on the telehealth portal as part of the ‘Digital Pakistan’ initiative was a short, knowledge-packed, and completely amazing journey that helped us learn and implement advanced quality assurance methodologies for a secure application. We adopted agile software quality practices to align software quality with product requirements and accelerate the software lifecycle. In addition, the continuous feedback we received from the project managers helped minimize retesting for verification and validation. Our iterative approach and short sprints enabled us to deliver quality products within a set deadline, successfully.

Thanks, team for all the amazing support. As I mentioned on slack, thanks to your hard work, we have 3000 doctors signed up and 1000 who submitted their documents. I had an amazing experience working with you all and truly admire your work ethic and efficiency. We couldn’t have done it without you. We will keep you updated on the stats and the launch event! — Project Coordinator, Digital Pakistan

Here’s where having a process-driven, and designed around ‘best-in-class’ software technologies delivery model helped us greatly.  It enabled robust scalability while maintaining cost-efficiency within strict quality control measures.

Thanks for stepping up to swiftly contribute towards our nation in these challenging times. It’s been a privilege to watch such a well-oiled team in action. — CEO, Digital Pakistan

---

HealthcareQuality assurance & testing

You might also like…