XForm and XLSFrom Standards are Preventing The Progress of Digital Health

Recently, apart from my work on FHIR and interoperability, I got a chance to work on clinical decision support, healthcare surveys and verbal autopsy. All these clinical decision support, healthcare surveys and verbal autopsies have one thing in common. All three struggle because of a long-forgotten web standard, a pseudo standard and applications that use these standards and do not break away from these standards.

Scenario

Let me take the most common scenario: a healthcare facility survey. Social workers or conductors of the survey go to the healthcare institute, with a tablet or a mobile device. They load the survey questionnaire from a remote server and then collect the data. The data gets submitted to the server online or collected offline at the site and then synced with the server later.

The survey questionnaire can be a simple survey with a static set of fields, or can be a complex questionnaire with hundreds of questions plus lots of skip logic, and calculations along the way. As developers, it won’t be a difficult task to create forms like this in JavaScript. But imagine having to develop something with no coding or HTML knowledge, and one institute can conduct more than one survey and might have the need to create a number of surveys per year. Plus in a low-resource country with little or no budget to hire developers.

The Standard

Currently, the defacto standard used to create forms for surveys like these is the XForms standard. And people use another standard called XLS forms. Where you can create forms with a very crude language, and add logic to the elements and group of elements with its own pseudo-language, which will generate the skip logic.

The logic in an XLS form looks like this,

if(${gender}='female' and ${p_age}>=24,41.2622536519346-2.2807946589665*${p_height}+0.0513162082758347*pow(${p_height},2)-0.000532227017487036*pow(${p_height},3)+0.00000267880129591717*pow(${p_height},4)-0.00000000502269777541732*pow(${p_height},5), 0)

Everything you do is based on Excel, and you will need to have the specific Excel columns and follow a strict pattern to generate the skip logic and correct form rendering. This XLS form can then be converted to XForms using tools like ODK (which I will talk about in the next section), and rendered on a device. This is a simple and intuitive way to create simple forms with no knowledge of HTML and JavaScript.

The way these tools collect and store data, also hinders how the data can be shared with other systems with using modern standards like FHIR. And with the lock-in these tools have on the data collection space, and their delay to adopt modern standards like FHIR also hinders interoperability and data sharing.

The Problem

The problem arises when the questionnaires become complex, with an increasing number of questions and skip logic. As the logic gets complex, the conditions increase in length and are difficult to manage, and everything snowballs into chaos

• No IDEs or tools. Everything has to be done by hand.
• Everything has to be in the same cell, there is no way to go to a different cell, or row as if you’re coding.
• No type checks or syntax checks, everything is done in Excel.
• Finding errors can become a nightmare and it’s like a needle in a haystack.
• Cyclic dependencies
• I’ve also seen some logic becoming so complex, it hits the character limit for a cell in excel.
• One small modification somewhere can crash everything and form from rendering.

Once someone generates a proper XLS file, it will be uploaded to one of the software servers mentioned below and gets generated into Xfrom which is used to collect data. Creating these files can take an unbelievably long time.

Sometimes WHO itself publishes the questionnaire in XLSForms and XFroms standard because they are widely being used in countries. Countries also find it extremely difficult to modify these forms for the reasons mentioned above, because XLSFroms are not a proper standard or a language. One has to come to the digital health landscape to see how this is slowing down the adaptation of proper standards and digital health.

The Software

Data collection tools like ODK, Kobo, and CHT are using these standards, and they have been established as the de facto tools for data collection in healthcare surveys.

And countries have been using these tools for a really long period, it has become really difficult to replace them with something better. As there are people who are used to collecting data using these tools, countries are reluctant to replace and train people with different tools.

The tools, even though they are open source, do not adopt proper interoperability standards, making it extremely difficult to exchange the data with other systems. The tools also do not seem to innovate and replace the XLSform with something better.

And it’s one of those niche corners where XForms are still widely being used and one reason for the existence of Xforms and XLSForms..

Finally, now there is a symbiotic relationship between these tools, and an old standard XForms and pseudo standard (XLS Forms). And one is kept alive because of the other. And there is no progress and in the field of health survey data collection. And even with all the issues, XLSFroms provides a semi no-code solution for countries to develop forms without the need to invest time and money on developers or solutions.

The Solution

In my research during last week, I was unable to find a proper replacement for XForms and XLSForms. I found one repository from 2016, that tried to replace XLSfroms with a JSON structure called JSON forms, but it has not received any update since 2016.

But what JSON forms did not understand is that what adds value to XLSfroms is the ability to add skip logic, and render forms dynamically depending on previous user inputs. Which has not been addressed in JSON forms. The ability to add logic without the need to learn JavaScript or HTML and generate complex web forms.

I think there should be a tool that allows people to generate web forms with skip logic, ideally a graphical UI, which outputs a standard HTML web form with JavaScript to add logic. I know there are survey tools that can add basic logic, but these forms have complex logic that can depend on responses collected at different stages of data collection.

These HTML forms can then be sent to a tablet or mobile device for data collection in the field.

In summery the solution should be a,

• A no-code tool that can easily create web forms.
• Ability to add complex logic depending on previous responses and dynamically generate forms.
• Ability to add this logic without coding in JavaScript with minimal syntax or GUI like people currently add logic in Excel.
• Push the form to a tablet or a mobile device.
• Collect the responses in a central server and share with other services to generate reports.

As someone working in digital health field, it can be really frustrating to see when we are living in an age of generative AI, and an abundance of really smart coders. But on the other spectrum, we tend to struggle to have a proper tool that generates forms for data collection based on real web standards.

I really hope one comes up with such a tool. I’d love to listen to your recommendations and feedback in the comments.