Adaptable Software

While computer hardware has changed drastically in the past few years, computer software has struggled to keep pace. The centralized, monolithic programming model that was adequate, when treating computers as isolated entities, is poorly suited to distributed, multi-task-oriented computing. For computing to become truly ubiquitous, new distributed, multi-task-oriented programming methodologies must be developed. We believe that distributed, multi-agent technologies offer the capabilities needed.

The Adaptable Software thrust consists of many projects. Each of these projects strive to investigate and develop systems that change their behavior over time due to some stimulation by or interaction with the environment. Three classes of adaptation have been identified in this work:

Parametric Adaptation

The variation of algorithm parameters over time. The change in an artificial neural net synapse weights as learning progresses would be an example of Parametric Adaptation.

Algorithmic Adaptation

The switching from one algorithm to another to solve a given problem. The alternation between various vision algorithms as lighting or other conditions change would be an example of Algorithmic Adaptation.

Resource Adaptation

The differing utilization of resources by a running distributed system over time. The relocation of executing code from one processor to another, presumably less encumbered, processor would be an example of Resource Adaptation.

Projects:

A Port-Based Adaptive Agent Architecture

Composable Skill Synthesis

Incremental Reinforcement Learning

Network/Traffic Systems - Routing and Path Selection

Task Decomposition

Back to the top

A PORT-BASED ADAPTIVE AGENT ARCHITECTURE

To facilitate the design of large-scale, self-adaptive systems, we have developed the Port-Based Adaptable Agent Architecture. This distributed architecture allows systems to be created with the flexibility and modularity required for the rapid construction of software systems that evaluate and modify themselves to improve performance.

Project Website: Self-Adaptable Software

Project Contact: Pradeep K. Khosla

Back to the top

COMPOSABLE SKILL SYNTHESIS

Machine learning methods have received much attention of late. Such methods allow an agent to learn skills that are difficult for a human to specify fully. To date, one of the greatest drawbacks to machine learning is that knowledge acquired while learning one skill does not enhance the learning of other related skills. It would be desireable for a designer to be able to compose an approximate solution using prior skill knowledge and then allow the agent to find a better solution via learning.

This work focuses on skill synthesis in a reinforcement learning agent. Reinforcement learning is unique in that an agent must actively explore its environment in order to learn. The prior skill knowledge is used to bias the exploration of the agent in such a way as to improve learning. The agent is even able to overcome the effects of poor skill selection.

Project Website: Self-Adaptable Software

Project Contact: Pradeep K. Khosla

Back to the top

INCREMENTAL REINFORCEMENT LEARNING

In incremental learning, a large, complex task is decomposed into smaller sub-tasks. If the task is decomposed properly then solving all the sub-tasks may be easier than solving the entire task, and by solving the sub-tasks, a solution to the target task is found. By easier, we mean that some objective measure of performance is greater than (or less than, as the case may be) the same objective measure resulting from using another methodology.

Project Website: Self-Adaptable Software

Project Contact: Pradeep K. Khosla

Back to the top

NETWORK/TRAFFIC SYSTEMS - ROUTING AND PATH SELECTION

Project Website: Self-Adaptable Software

Project Contact: Pradeep K. Khosla

Back to the top

TASK DECOMPOSITION

Project Website: Self-Adaptable Software

Project Contact: Pradeep K. Khosla

Back to the top