Official Atlas Battleplan

As you all have probably noticed, as Atlas is getting hammered out and starting to shape up, some concerns have been voiced over various perceived or potential issues. This Battleplan is provided to serve as a guideline for how issues such as these can be verified, investigated, and integrated, in a way that achieves our collective goals in the most efficacious manner.

This Battleplan derives from the planning work done at an impromptu brainstorming session two days ago on irc.

Mission Statement

Atlas provides a standardized system for conducting networked communications between clients, servers, database apps, and other such tools and utilities, specifically for communications necessary to establish and interact in computerized multiplayer roleplaying games, realtime strategy games and other online virtual environment simulations.

Motivation

WorldForge encompases a number of different client, server, and utility tool product development efforts. In order to build and operate a networked game, multiple products must be employed - which means they must be able to communicate with each other. In order to maximize the range of product choices available to system integrators, the ideal situation is if all products are able to communicate in a manner that most or all other products understand. Thus, a standard, well-defined, common protocol that all of these applications can use, is in all of our common interest.

However, it is unlikely that perfection can be achieved right off of the bat - either in the protocol itself or in the various implementations of it. If a means of incremental improvement is not available, then individual developers may be forced to adopt their own independent and incompatible protocols for their work. Furthermore, it is unlikely that the common protocol will include every special feature needed by each particular developer; thus an extension mechanism must be made available within the protocol's definition itself. The danger of not providing this is severe: As developers choose to ignore the standard, it degrades in usefulness, and once a sufficient number of developers shun it, the entire concept of supporting a standard protocol is undermined and collapses.

This Battleplan provides a structure that will help define for developers how to work within the Atlas System while still being able to achieve their own development intentions. As well, it encourages development of Atlas to occur by the "patch submittal" process typical to most other open source library development projects. Finally, it outlines a developmental roadmap that can be followed by developers in the near term, to simultaneously further Atlas growth, usage, and perfection.

Atlas Architecture

An Atlas System encompases several discrete and independent elements. The independence of these elements is included in the system architecture for the express purpose of allowing for future swapping out of one implementation of an element for another, while preserving compatibility with other elements. These elements are:

ATLAS PROTOCOL: This is the general rules of procedure for how to communicate across a network using the Atlas System. The Protocol is independent of any network transport or stream format. The protocol specifies the sets of commands needed to carry out various actions either out of game or in game. The Atlas Protocol Standard is maintained by the WorldForge Atlas Protocol Coordinator; all proposals for changes to the Atlas Protocol must be reviewed by the Protocol Coordinator for acceptance into the standard. The Protocol Coordinator is responsible for incrementing the Protocol Version Number as appropriate.

ATLAS TRANSPORT FRAMEWORK: An implementation of the protocol either on the client side or server side, is known as a Framework. A Framework manages all aspects of network communication, including the connection, encoding/decoding of messages, and the transport of data from the main app through the encoder/decoder, to the network communication subsystem. Different Frameworks may exist in different programming languages. Frameworks may either be designed around a single stream codec and network connection library, or may be designed to include multiple "pluggable" codecs for flexibility. Frameworks are not required to be compatible at the API level, however they must communicate with at least one of the Registered Stream Formats in order to be considered an Atlas Compliant Transport Framework. Frameworks must be capable of transmitting the Atlas Protocol Version Number as well as the Stream Format Identification Code for purposes of protocol negotiation, as described in the Atlas Protocol Specification

NETWORK CONNECTION: This is the underlying networking scheme used to transfer data from one location to another. For example, TCP, UDP, and MCP are Network Transports.

NETWORK CONNECTION LIBRARY: This is a particular piece of code that allows use of a particular Network Transport. For instance, TCP4U, WinSock, and UnixSockets are Network Connection Libraries.

STREAM FORMAT: This is the network representation of the protocol's data. ALL OFFICIAL STREAM FORMATS MUST BE NAMED AND REGISTERED WITH WORLDFORGE.ORG. Registration of a Stream Format requires a complete set of tag documentation, specification of a name, and a A.B.C style revision number. Stream Formats must be capable of representing at least one version of the Atlas Protocol, which must be indicated in the documentation of the stream format. The only Stream Format registered presently is AtlasObjXML, which is now at version 0.2.1. It is expected that others will produce a Humanized XML, Packed ASCII, Binary, and ASCII Encoded Binary formations.

STREAM FORMAT CODEC: This is an implementation of a particular Stream Format in a linkable module. While there is no "standard" Codec API format presently, in the future it is hoped that a standard Codec design will emerge.

Operational Description

In practice, the Atlas system sits alongside the game engine. The game code is responsible for assembling Atlas message objects, and submitting or receiving these from clients or servers. For instance, the game client will establish a connection to a remote server host, negotiating protocol using a mechanism provided by the particular Atlas Framework, assembling an Atlas message object to establish a login (storing the information provided about the character locally), and creating and submitting atlas message objects to conduct various operations.

Reference Implementation

Working examples of Atlas will be provided as baselines against which new ideas can be tested and measured. As improvements are discovered they can be incorporated into future versions of the reference implementation, or can replace it altogether. The authority to determine the development plan for the reference implementation rests with the Atlas Protocol Coordinator or a representative of his choice.

Currently, the reference implementation is libAtlasPy, a Python library written by Aloril that uses the AtlasObjXML-0.2.1 Stream Format. Use of libAtlasPy is illustrated in Cyphesis.

Due to the fact that many developers prefer a C++ version of the library over Python, the reference implementation of Atlas will switch to a complete C++ implementation. Because of the desperate need to get this now in a fashion that works properly, we will use an updated version of Sebastien's tested libAtlas Framework (currently in CVS under forge/protocols/atlas/libatlas/) which includes use of TCP4U as the Network Connection Library and is being modified to use the AtlasObjXML-0.2.1 as the Stream Format. As soon as it has been verified to work as well as the previous reference implementation, this will become the official standard, and developers will be encouraged to make use of it.

Because of the desire for "swappable" Stream Formats, more detailed and explicit documentation, and need for a "Next Generation" design, we will open the floor for "from scratch" development of a new reference implementation. This new reference implementation must provide all of the following documentats and features in order to be considered:

  • Requirements / Specification for Library Detailed Design Documentation of Library Easy to Read and Follow Tutorial for Using Library Complete User Reference Documentation for Library Implementation of Library Buildable on Linux and Windows Demonstration Game Implementation Test Suite Implementation * Completed Verification Report

Test Plan

Obviously, in order to determine if a given implementation is working correctly in relation to documented standards as well as other implementations, we need to have a set of standard verification tests that can be run as needed against any given implementation. Milamber is generating this test plan, and hopes to have it within a week.

Benchmarking tests will also be useful for purposes of contrasting two otherwise equivalent implementations.

Development Efforts

In addition to the reference implementation work, several other potential development efforts have been identified. Organization and coordination of each of these efforts is left to the responsibility and authority of the developer(s) interested in them. Below is a non-comprehensive list of potential efforts:

Documentation: This important effort centers around the immediate and overwhelming need to have more thorough and detailed documentation on the current reference implementation. This documentation will also be useful for the next reference implementation.

Cleanup: The current reference implementation uses threading. This effort would be aimed at modifying the library to remove this scalability limitation. Also, the current version is implemented by means of several discrete libraries, which might be worth combining into a single library.

Pong: This game development effort aims at providing a working demo of the current Atlas reference implementation. Bloodsport is the lead on this effort.

libminiXml: Some development work has been identified for this component of the current reference implementation. For instance, the Msg class has proven difficult to use and may need replacement. Sebastien is considering this effort.

New Atlas Architecture Development: This effort is being led by ZW to create a new reference implementation architecture and implementation that supports modularization of Stream Format Codecs, allowing selection of Codec to be determined at run time via negotiation with the remote. It is a "from scratch" development aimed at avoiding dependencies and at providing a very flexible design.

New Stream Formats: Several concepts for additional stream formats - including Binary, Compressed ASCII, "natural" or "human readible" XML, and others, have been mentioned, along with indication of desire to develop these formats.

Roadmap Summary

Below is a roughly ordered list of Atlas-related product releases expected to occur over the next few months. No schedule is provided because the release dates are indeterminate.
  • Atlas 0.2.1 Release Battleplan Release Atlas 0.2.2 Release (incl. Mondo Documentation) libAtlasWF 0.2 Reference Implementation Release Atlas Test Plan Release Alternate Stream Format Development Releases Atlas Demo App Release(s) New Atlas Architecture Adoption Atlas 0.2.3 (MIM Integration?) Atlas 0.2.4 (Media?) Atlas 0.3 Protocol Release (3-6 months from now)

Conclusion

It is hoped that this battleplan provides sufficient structure to keep Atlas development proceding consistently, while allowing sufficient flexibility to developers wishing to experiment with new ideas. It is hoped that it is also clear the benefit to cooperating on development of this standard, and the crisis of fragmentation that could occur if WorldForge people do not try to stick together on this.

Showing commitment to following or supporting this Battleplan will do a great deal of good in mending the splintering that is already going on, as well as encourage others to focus efforts into getting Atlas in place and in use as soon as possible.

Aloril is hammering out a great deal of Atlas documentation, adding much detail to what was already there and producing example cases. He is going to also use Cyphesis to generate complete scenarios for various activities. Keep an eye on website/protocols/atlas/ for new documentation.

Cyanide has gathered the libAtlas-0.2.0 Requirements Document, which will be reposted for discussion and deliberation for use in developing future Atlas Frameworks. Please provide your feedback for this document on the scripting@worldforge.org mailing list.

The official forum for discussion of Atlas-related issues and development can be subscribed to by sending a blank email to scripting-subscribe@worldforge.org.