We characterized the domain-independent challenges posed by an execution aid that interactively supports humans monitoring the activity of distributed teams of cooperating agents, both human and machine. The most important issues for interactive monitoring are adaptivity, plan- and situation-specific monitoring, reactivity, and high-value, user-appropriate alerts. We showed how properties of various domains influence these challenges and their solutions. We then presented a top-level domain-independent categorization of the types of alerts a plan-based monitoring system might issue to a user. The different monitoring techniques generally required for each category are often domain specific and task specific.
Our monitoring framework integrates these various techniques and then uses the concept of value of an alert to control interaction with the user. This conceptual framework facilitates integration of new monitoring techniques and provides a domain-independent context for future discussions of monitoring systems. We discussed various design tradeoffs that must be made during the application of our monitoring framework to a domain (Sections 6.4 and 6.6).
We use this framework to describe a monitoring approach we developed and have used to implement Execution Assistants (EAs) in two different dynamic, data-rich, real-world domains. Our approach is based on rich plan representations, which allow the execution aid to filter, interpret, and react to the large volume of incoming information, and alert the user appropriately. An expressive plan representation is necessary for representing SUO plans, which must coordinate distributed units, trigger contingencies, and enforce a variety of constraints. It is equally important that this representation be monitorable by machines and meaningful to humans. Our plan representation and mission model were able to model a representative SUO scenario with enough fidelity to provide value (as judged by our domain experts) and was also sufficient for plans in the UV-Robotics domain.
We developed a sufficiently rich plan representation by extending an existing plan representation with a hierarchical, object-oriented mission model that encodes knowledge about primitive actions and mission-specific monitoring methods. The SUO EA implements a novel integration of these hierarchical monitoring methods with a reactive control system. The EA invokes the most specific methods defined in the hierarchy at appropriate points during monitoring.
One central challenge, in our domains as well as medical monitoring, is to avoid overwhelming the user with unwanted alerts and/or false alarms. We define the concepts of value of information and value of giving an alert as the principles for determining when to give an alert. We describe the properties of VOI and VOA, criteria for computing them, the advantages of qualitative reasoning in our domains, and the successful use of these concepts in our applications. VOI and VOA algorithms must be customizable to the user, plan, and situation.
By using an asynchronous multiagent architecture and an extended version of the PRS reactive control system, we monitored the execution of both SUO and UV-Robotics plans with acceptable latency, given a dozen or more incoming events per second. PRS extensions include temporal monitors and efficiency improvements. Methods from the mission model are used throughout the SUO monitoring process for action-specific monitoring. Our evaluation showed that our plan-aware EAs generated appropriate alerts in a timely manner without overwhelming the user with too many alerts, although a small percentage of the alerts were unwanted. We have shown the utility of using advanced AI planning and execution technologies in small unit operations.
The application to UV-Robotics showed the generality of our SUO framework and monitoring concepts. We implemented a complex execution assistant in about one person-week, using code from the SUO EA. The UV EA uses the same plan representation and basic architecture as the SUO EA, but the inputs are different as are the tasks and the algorithms that respond to the inputs and generate alerts.
Future work. The most obvious area for future work in the SUO domain is incorporation of a planning assistant to complete the loop of continuous planning and execution. This integration has already been accomplished in the UV-Robotics domain, but the difficulty in the SUO domain is an interface that allows a soldier to interact effectively with the planning tool, using a wearable computer in a battlefield situation. Several research programs are addressing this problem, some of which are mentioned in Section 2.
Within the scope of execution monitoring, future work on our EAs could model and detect other types of plan deviations (such as loss of surprise or additional types of fratricide risks), project future failures, and provide higher-fidelity specialized reasoners, particularly for terrain reasoning. Additional theoretical work on VOI and VOA would support better quantitative estimates of VOI and VOA. The SUO mission model already has a method for projecting failures and a low-fidelity projection capability could be easily added. In the UV-Robotics domain, we plan to implement additional types of alerts in the near future, and extend the UV EA to serve the higher layers of the architecture that have more in common with the SUO EA alert types and triggers. The fragility of the UV communication network in hostile domains provides a set of interesting monitoring challenges that may result in the incorporation of specific monitoring-related tasks within cooperative team missions. Monitoring strategies for uncertain communication environments is an important research challenge for the UV-Robotics domain. Additional alerts being considered for future implementation include monitoring movement of entities in and out of geographical sectors mentioned in the plan, monitoring the deterioration or improvement of communication conditions, and monitoring the actions and intentions of coordinating team members to facilitate cooperative behavior.