Once upon a time human computer interaction was mainly about using some GUI (Graphical User Interface) through a computer or terminal. Over the years, more interaction modalities were developed based on technologies like mobile computing, Natural Language Processing (NLP), speech interfaces, touch interfaces and extended reality (XP). These interaction modalities led to a significant improvement to the overall User Experience (UX) and enabled the creation of personalized and ergonomic user experiences. In recent years, there is also a trend towards combining two or more of the above listed modalities in the scope of a single application and interaction experience. This combination is enabled by a new wave of multi-modal interaction platforms, which are conveniently called multi-experience platforms.
Multi-Experience platforms open new horizons to user experience, through complementing mobile applications and mobile experiences with conversational, immersive and IoT (Internet of Things) characteristics. In this direction, multi-experience platforms enable the combination of mobile apps, conversational apps, immersive apps and IoT-enabled apps (e.g., wearable applications) in a single system. The combination of these applications enables the development of novel, personalized digital products that provide a consistent experience across different channels and touchpoints. In this way, they also maximize end-users’ satisfaction.
Technology Enablers of Multi-Experience Platforms
Multi-Experience platforms combine a variety of different technologies. They provide the means for integrating and combining these technologies towards uniform and consistent experiences to end-users. Some of most common technology building blocks of multi-experience platforms are:
- Mobile Computing and Mobile Apps: Most of the users’ interactions with digital products and services take place through smartphones, tablets, and other mobile devices. This drives the so-called mobile-first computing trend, which refers to the prioritization of mobile computing as the primary interaction modality. In this context, multi-experience platforms provide the means for developing, deploying, and hosting mobile apps.
- Natural Language Processing (NLP) and Conversational Interfaces: A significant number of google searches in 2020 took place through voice interaction. Likewise, the popularity of services like Siri, Alexa and Hey Google is increasing. Moreover, voice platforms are integrating in virtually every digital service such as e-banking and e-commerce services. Also, novel conversational modalities are under development such as advanced spoken dialogue interaction. In this context, NLP and conversational interfaces are becoming versatile and effective tools for the development of multi-experience applications.
- Immersive Interfaces and Extended Reality (XR): There is a growing interest on Augmented Realty (AR) and Virtual Reality (VR) cyber-representations of virtual and real worlds. AR enables users to interact with the real-world based on the concept of tactile intelligence i.e., almost in real time. Also, VR facilitates the development of realistic simulations in virtual settings. VR and AR applications are increasingly combined in the scope of sophisticated applications that are characterized as Extended Reality (XR). XR provides immersive experiences in various application domains such as healthcare and industry. In the years to come, the number of immersive experiences will proliferate because of the increased capacity of communication networks and the falling prices of XR headsets. Hence, modern multi-experience platforms will provide support for immersive interfaces and XR interactions.
- IoT and Wearable Applications: In many cases, end-users interact with state-of-the-art applications using devices like smart watches, smart wristbands, fitbits, and other wearables. Such interaction modalities are very common in application areas like fitness, healthcare and lifestyle management. Multi-experience platforms support interactions with IoT wearable devices, including collection of data from them and visualization of insights within their user interfaces.
- Multi-modal Interactions: Multi-experience platforms integrate multimodal interaction capabilities. The latter enable end-users to interact with an application using one or more of the above modalities, without any loss in the context and functionality of the application. Multi-modal interactions are enabled by common data models of the user interaction which are shared across different channels and modalities. The data models remain up-to-date and synchronized regardless of the interaction channel used by the user. Alternatively, there are also multimodal applications that are authored based on multimodal markup languages e.g., markup languages that combine mobile, immersive, and conversational interactions.
Multi-Experience Platforms Development Trends
In coming years, multi-experience platforms will integrate novel ICT technologies to boost the automation, intelligence, and consistency of the UX that they deliver. Specifically, the following trends are envisaged:
- Automated Synchronization of Modalities using AI: Multi-experience platforms will leverage Artificial Intelligence (AI) to automate and personalize the multi-channel experience of the user. For instance, AI algorithms will be employed to discover the users’ preferred channels and interactions, leveraging historical information about past actions of the users.
- Redefining and Augmenting Legacy User Experience: In the medium-term legacy applications will be enhanced with multi-experience features. For instance, mobile applications will be enhanced with speech enabled capabilities and immersive interfaces. Moreover, multi-experience platforms will provide the means for integrating emerging technological capabilities such as interactions across sensors, edge devices and IoT devices. Machine learning and AI techniques will be used to enable the selection of the most relevant interaction modalities, along with a seamless switching between them.
- Trustworthiness and Explainability: To engage in the use of multi-modal, multi-experience interfaces, users must be provided with full transparency on how the different interfaces operate and complement each other. This transparency will be foundational for the user acceptance of the multi-experience platforms. To enable transparency and trustworthiness, multi-experience platforms are likely to integrate explainable artificial intelligence functionalities.
- Speed and Agility: Multi-experience development platforms will incorporate agile development functionalities, such as automated development, integration, and testing, as well as one-click deployment features. This will enable enterprises to adapt the multi-experience business logic to changing requirements regarding interaction channels and user-interfaces. In the future, the interaction channels used by an enterprise will be an element of the enterprise’s branding strategy. Specifically, enterprises will opt to deliver their messages through the combination of channels that they consider most appropriate to their brand.
The digital experience of modern enterprises is delivered through multiple channels and modalities. Multi-experience platforms provide the means for orchestrating and synchronizing these channels, in-line with the business, marketing and branding requirements for digital products and services. This is the reason why it is time for digital firms to adopt multi-experience platforms that will help them deliver consistent, effective and personalized experiences to their customers.
