Road Map
From Croquet Consortium
Foreword
Each element of the system is addressed here with a description, current status, the gatekeeper, and a rough indication of priority, either fundamental – e.g. essential to the bootstrapping process, important for the short term, or a long term project. Additional resources can allow an element to be promoted from a longer term project to a shorter term. Each element indicates who has current primary responsibility for the project. All of these elements are open to comments and ideas from the community, but even more important, we welcome coding resources. It is one thing to discuss how something should or should not be done, but it is much more interesting to play with working code. This list is not in order of importance.
Croquet Roadmap
Version 2.02 May 17, 2007 The Croquet Architecture Review Board
(Q) - means that Qwaq, Inc. has developed a proprietary solution to the task. In such cases, Qwaq personnel would probably not be in a position to act as gate keepers unless Qwaq decided to move the solution to open source.
There are four levels of priority:
- critical - this item is an essential part of Croquet. Without it, Croquet would be seriously crippled.
- high - very important for the definition of the system.
- moderate - reasonably important. Immediately useful, but there are either work-arounds or it is seen as a critical or high priority.
- low - nice to have, but not necessary to the definition of the system.
Note that there are three critical items: Island Classes, Two Dimension Infrastructure, and Documentation.
Tea Time
Description The TeaTime architecture is the basis of true scalable collaboration in Croquet. As such it is a central part of the very definition of Croquet. The current incarnation of TeaTime relies on a "micro-server" called a router that is the central authority for determining timing of external events. We need to explore the true peer-to-peer architecture that David has designed. Potential benefits would include a more robust scaling architecture eliminating a single point of failure.
Use of global advertisement/naming mechanism for CroquetPlace? advertisements. Write-barrier based object versioning to support temporal reflection. Hooks for debugging based on causal history. Ability to slow progress of time across multiple users (multirate clocks).
Status Working systems have been developed. Probably need to reconsider these in light of the Hedgehog architecture - using this as a well-defined bottlenecked system. Simply replace the current router approach with a p2p system. Islands do not have any information about the nature of the replication model.
Priority Moderate
Gate Keeper David Reed
Island Classes
Description Island Classes are classes that are part of the replicated Island's definition. They can be edited like any other class, but only exist inside of a particular Island. The code can be reused only by copying the classes between Islands. This goes beyond a uni-class framework, turning Islands into complete encapsulated development environments.
Status Discussions.
Priority Critical
Gate Keeper Andreas Raab
Two Dimension Infrastructure
Description This is an integrated 2D architecture that allows the user to easily and quickly create hybrid 2D/3D applications in Croquet. In short, these 2D objects should be able to exist in the 3D environment with no changes or support, and act full 3D objects with no additional support. This also requires a modern imaging capability that leverages the hardware where possible. Another key element is that these objects must be replicated. This means that any 2D operation, such as text editing, must work perfectly in the Croquet collaborative environment.
Two possibilities that we need to examine in this context are:
- Cairo – This is a modern imaging layer for the Xwindows system, but looks to be useful in a high-performance OpenGL environment.
- Tweak – a new 2D development/scripting architecture.
Status Tweak has made an appearance in Croquet and is currently being developed as a parallel effort. Cairo is a bit further off and is a pure imaging library layer with little support for more complex operations and needs. Very possible that Tweak may make use of Cairo at some point.
Priority Critical
Gate Keeper Andreas Raab
Coding and Debugging
Description We need a fully integrated, collaborative coding and debugging facility and a powerful interface to it. This would be built on top of the 2D infrastructure described above. We should consider rethinking the current browser to make access to the code a bit less intimidating. Certainly, scripting interfaces will help with this, but there is a real need to categorize the code by the nature of its use and expected user.
Status Preliminary
Priority Moderate
Gate Keeper DAS
Instant Messaging Jabber (Q)
Description Instant Messenger clients can act as a basic “dial tone” for Croquet - and helps establish that your buddies are online before attempting to launch a croquet session. We can also send XML-encoded postcard info (or URLs that point to postcard info) so IM can act as an ad-hoc rendezvous service for those who do not run maintain rendezvous services of their own. This means that a Jabber client in the native OS should be able to accept “meet me here” information that can be handed to a Croquet client to launch a session.
At some point, we see this turning inside out, where Croquet is always the primary interface, but I expect that this will take some time. This means that the Jabber client may require two kinds of interfaces, a more traditional one as part of the user’s legacy desktop, and a Croquet variant where it is integrated with the 3D world.
We can also leverage Jabber as a text only way for people to participate in a croquet space by mapping croquet spaces to jabber chat rooms and echoing text chat between croquet-only participants and jabber-only participants.
Status The Jabber client exists inside Croquet and , we have defined XML encoding of postcard information to rendezvous. More work needs to be done on a clean way to establish an interconnection between a given croquet space and a jabber chat room. The basic functionality described here works, but needs some cleanup and documentation.
Priority High
Gate Keeper MM
VoiceOverIP (Q)
Description Voice over IP works, but needs refinement in several areas. Echo cancellation for the current croquet 3D audio would be nice to have but is not crucial if we assume most people use headsets. We should also provide an example of non-3D audio to enable private conversations (where only selected participants can hear what is being said rather than everyone in the space).
We also need to put some work into interoperating with standards for VoIP so that non-Croquet-enabled people can call into a space (on a virtual speaker phone) and participate in the world. Using the Asterisk open-source software as a bridge should be investigated here since this could get us a bridge to the public phone system.
Status 3D voice chat works today. Requires a rewrite(or a bridge) to incorporate standards and reach out to the rest of the voice world
Priority High
Gate Keeper MM
VideoIntegration
Description Both simple live video streams for mixed-reality spaces and avatars represented or enhanced by realtime video projections will aid the collaborative sense of the system. Finding a way to use commercial codecs for video in an open source framework may present some challenges unless we leverage some of the native platform support provided by things like Quicktime - but this may raise thorny issues in the Linux world. Status Demo-ware quality at this point for Macs and PCs. Howard Sterns and Preston Austin need to locate the plugin source for the video demo.
Priority Moderate
Gate Keeper MM
AnnotationArchitecture
Description User interface elements for creating and retrieving annotations of objects in the Croquet space currently exist in Hedgehog, but these annotations do not interface to a digital repository - which is vital to move this beyond demoware. Annotations allow users to associate objects with existing objects in a scene and is a way for users to rapidly develop context-based content as well as selectively hide/reveal users’ comments/annotations on a virtual space. The most important work to be done here is to integrate the current annotation user interface with a shared repository so that there is a community-wide platform for comment. To associate an annotation with an author we also have to deal with the user-identity issue for the croquet community. See the Worldbase section for more on these issues Status demoware capability exists today - needs integration with a repository to be generally useful.
Priority Moderate
Gate Keepers Mark McCahill
NavigationControl (Q)
Description We note that a number of users accustomed to game navigational conventions are befuddled by navigational controls in the Jasmine release. The whole three button Squeak mouse thing doesn't seem to work at all for most users. We definietly need a better way to get around. We are exploring alternative ways of navigating in Croquet places.
Status Several conventions for keyboard/mouse navigation have emerged in the 3D gaming world and we are treating these as a starting point for the navigational interface of Croquet implementations intended for use in educational settings. Doing so will accommodate existing 3D-savvy users and reduces the potential for frustration until a more appropriate solution can be devised and developed.
We find that Doom III’s navigational interface design is a good starting point. As in many other first-person-shooters, controlling movement and selection within Doom III requires only a mouse and keyboard - and all modern computers have these already. Arrow keys and the “WASD” keys can be used to move about the environment. Use of the arrow keys is an intuitive way to control movement within the space and redundancy with “WASD” keys will allow right-handed users to use their left hand for navigation while using their right hand for mouse-look and select functions.
Some of the literature on navigation in 3D environments shows that users who are unfamiliar with game conventions presume the mouse is used for moving around the space. We feel that the solution to this conflict in user expectations is to set the navigational defaults to keyboard control and then provide user-configurable preferences to accommodate experimentation with mouse-based navigation.
We are presenting one version of a navigational scheme at the C5 conference in Kyoto on January 28th
Priority High
Gate Keeper Peter Moore
Physics
Description Open Dynamics Engine. This is an open source physics engine that we are using inside of Croquet. It needs to be modified such that it will execute in a deterministic manner and needs to be integrated with TeaTime.
Status It has already been ported to all three major platforms.
We're working to come up with an approach to enable Croquet-based simulations involving ‘random’ events to be replicated on multiple machines. This is an interesting problem because of the fact that replication of computation would also replicate generation of the random numbers to be used in a simulation. This would usually be undesireable. For example, if we create a virtual coin toss simulation in Croquet that is intended to produce a random outcome (the outcome of either "heads" or "tails"), we would have to ensure that what might come up as “heads” on my instance is also “heads” on all other clients in the TeaParty. The problem is that if all of us are replicating the calculation of a random outcome, there would be nothing to ensure that what appears as “heads” on my machine would not be “tails” on yours. Solving this problem has deep implications on making the somewhat non-deterministic ODE (Open Dynamics Engine) integration with Croquet both deterministic and collaborative. Setting aside the obvious philosophical treatise that the act of structuring a form of deterministic randomness in cyberspace might warrant, we're presently considering two practical approaches to this issue.
One approach is to generate results on a master machine (the initiator of the event) and then replicating the random number computation (actually a pseudo-random number computation) on all other machines in the TeaParty (one machine simply tells all the other machines in the TeaParty that a new final result has been produced). In this way, when the triggering event or gesture happens on a particular machine, then that machine consults its own random number generator to produce a result. When the same event or gesture occurs on another machine, the random number generator on that machine is consulted to produce the result. For moderately complex simulations, there will be little guarantee of uniformity across ones that are moderately shared.
Another approach would be to get all participants to behave as if they are using the same random number generator. This would involve creating one random number generator that produces a well-distributed sequence of random values, regardless of which machine originates the request for any one of these values and the resulting sequence would be shared by all members of the TeaParty. In so doing, each request for the next random number would be shared as a TeaParty-wide? event by all participating machines and no machine would be permitted to request an additional random number on its own that is not seen by all the other machines. This does not necessarily mean that there would be one generator for all purposes in all simulations running in the TeaParty. There could be one shared generator per ball in a billiards simulation, or one shared generator per billiard table in a simulation, or even one per TSpace. The point being that each of these shared generators is individually a single conceptual generator that is shared by all machines participating in the TeaParty.
Priority Moderate-High
Gate Keeper DAS
OpenAudioLibrary (OpenAL) (Q)
Description Integration of the OpenAL 3D sound library into Croquet. Would be great to have some VM support for objects with data buffers that are not relocated (which we would also like for textures) so that we could avoid copying the sound bits from the Croquet memory space to the OpenAL memory space.
Status We have OpenAL plugins for Mac, Linux, and Windows for 3D sound played in looping mode (same sound repeating over and over as in a helicopter or a bird call) and for live streams such as microphone input. It would be nice to do echo cancellation for the microphone input, but that is not a show stopper. As we look into better integration with standard VoIP we will probably have some work to get those sound sources integrated.
Priority High
Gate Keeper MM
TasksInteractorsReplicants
Description Tasks are replicated action objects that are generated by the users actions. These objects consist of an initialization phase, an iteration phase, and a termination phase.
Status Island-based tasks now exist as part of the core system. Further work needs to be done on internal Island tasks/interactors/replicants.
Priority Low
Gate Keeper David A. Smith
ApplicationFilters
Description Filters are a kind of portal where the image of the TSpace on the other side is augmented or changed in some way. Further, a filter can act to modify a users actions to modify the results of a user action.
Status Under design. Prototypes have been developed.
Priority Low - until something like Wicket gets some momentum.
Gate Keeper David A. Smith
WorldBaseContentServers
Description Manage the storage and retrieval of persistence data sets, and objects that can be shared and re-instanced throughout the Croquet world. Ultimately provide a federation of distributed WorldBase servers to handle a large distributed user population. Need to provide for attributed authorship of objects so we need to integrate with authentication infrastructures. It should also be possible to browse and search the worldbase by object meta-information.
Status A simple WorldBase server running on Linux as a quasi-web service so storage/retrieval of objects was developed for the Jasmine release. This demoware version needs to be made much more scalable and robust and ported to Hedgehog so we have a digital commons spanning the puiblic croquet spaces. The basic query/search functions need to be expanded and we need to provide gateways to make other object repositories to appear in croquet via the worldbase interface.
Unique IDs for Worldbase objects are generated from SHA1 hashes on the object and it’s metadata, so we can compose together annotations that have been independently created if they refer to the same base object. We need to add some options for tagging the license terms under which the objects are made available (creative commons is our friend here). Need to add some capability of keeping objects cached in the worldbase from being a vector for spreading nasty executable code. Need to integrate with existing identity systems so that there is attribution of items in the worldbase to users in the real world - higher ed will care about shibboleth and federated identity management systems here.
Priority Moderate
Gate Keepers Mark McCahill
CapabilitiesSecurity
Description Internal security utilizing capabilities based approaches.
One aspect of Croquet security is external security, aka PeerToPeerSecurity, such as encryption of communications using SSL. There are a host of institutional requirements for managing security and interoperating with middleware (e.g., at our university) that will involve compatibility with internet2.edu MACE standards such as signet.
Another aspect is making the internal distributed object model sound. The plan is to implement a foundation in the form of a distributed capability-based infrastructure (meaning if you have a reference for an object you can send it a message, objects choose which messages to understand, and you limit access by limiting propagation of references).
However, that doesn't solve the security problem - it just provides a solid foundation. The real security problem in Croquet doesn't fit the classic "document security" model that comes from focusing on file systems and databases too heavily. We presently don't know precisely what the word 'security' means for Croquet. However, we intend that whatever it ends up being, it will not be incompatible with Shibboleth (which is more of a WWW browser-based technology).
The hope is that a capabilities based approach for the later will allow modeling of not only our own institutional environment, but also very different environments in ways that will allow them to interoperate in a consistent way. Examples of other environments might include military environments (which have a different model than I2) and global public anonymous environments.
It is important to note that whatever approach is taken, all this must work between mutually suspicious objects on mutually suspicious machines, as well as mutually suspicious objects on a single machine.
Status Facet-based architecture is part of Hedgehog. It has not been utilized to its full extent. This will require additional developer and admin controls.
Croquet and Signet:
The U.Wisconsin is a member of the Internet2 MACE Signet Early Adopter program. Signet is a tool for managing fine-grained authorization and role information. The Signet Working Group is led by Lynn McRae? at Stanford University and seeks to explore a privilege management system from Internet2 MACE Signet. Their approach seems well suited to a P2P world, and signet software is now available.
UW and Signet development teams are now in a collaboration centered around privilege management in a VO (that's "Virtual Organization" in the sense of the term popularized by the Grid community) deployed within a Croquet environment. Signet appears well suited to managing fine grained permissions on objects in distributed environment such as Croquet. Involvement between Signet and Croquet Project efforts will highlite new areas of work for Signet because of its decentralized, peer-to-peer model and its unique provisioning challenge--how privilege information infrastructure can be extended to help manage users' access to objects and their services in Croquet space.
The first thing we are focused on is to explore mapping Signet privilege delegation trees and lattices to a Croquet-based peer-to-peer community. Imagine a collaborative educational effort that brought the subspaces of several participating developers at multiple institutions into a single Croquet world. I guess you can call this a virtual organization (VO). Signet would have to treat each developer in the VO as a root authority for permissions on objects they create, leading to an array of relatively small and short privilege trees when compared to a privilege management systems that covered financial, organizational and academic hierarchies across a single large research university. In addition, there is a limit to scale when permissions have to be granted to individuals one by one. As the VO grows, the authors of services and resources will eventually find it necessary to develop a shared VO-wide vocabulary of roles and rules in order to keep privilege granting manageable. The Signet team recognizes this as an area for future work. The Croquet project will now provide one driver for that effort.
Retrofitting into an existing memory-safe language:
In order to solve the intra-machine problems, see The Oz-E Project. The effort ahead to create a secure Squeak-like language will be both easier and harder than the effort ahead for Oz. It will be easier because Squeak is already fully object oriented (of course). It will be harder because the existing Squeak system has a much greater reliance on global namespaces and global mutable state.
Inter-machine Problems:
Regarding inter-machine problems, we have yet to understand how the current TeaTime architecture can fulfill these requirements. "Mutual suspicion" is a different threat model than the Byzantine threat model, and so Byzantine Paxos or similar algorithms don't address the real issues.
Priority Low
Gate Keeper Andreas Raab
Graphical User Interface
Description Since everything in Croquet is fully modifiable, it is possible for many different GUIs to be developed as applications are built. However, we still have to start somewhere. The framework for both using and customizing the user interface needs to be developed from an end users perspective. This is important for end users (not just programmers) to begin working with these powerful technologies/capabilities in the coming year. A growing community of users will fuel a growing community of developers and will attract resources in support of the refinement and use of Croquet. It is also important for Croquet to present an attractive initial GUI that reflects the quality of its underlying technologies. We are therefore designing and developing the first iteration of a simple and familiar default interface for Croquet - one that can allow technologically naive people to quickly and easily access the power of the underlying technology. The idea is that, through a simple and familiar interface, end users will be able to access the power of Croquet technology easily and efficiently and develop compelling applications and simulations (and with out the need to program in Squeak). The GUI can be used as a starting point for the development of better and more functional implementations. In this way hope to truly open up the power of this technology to the broadest collaborative user base.
Status We are working on:
- Event handling model
- Tweak integration
- Usability design (metaphors and user workflow)
- Menu design
- Panel design/functionality
- Usability testing
Next release:
- Integrated help (via scamper browser)
- Scripting integration (TVML)
Future releases:
- 3D monitor support
- Force feedback
- Gesture support
Priority High
Gate Keeper Liz Wendland
LanguageSupport (Q)
Description Croquet should support multiple languages. We have already integrated the Babel architecture, which supports a number of important first class object-based languages. We also need to examine integration with Java and look at perhaps replacing Slang with an on-demand C compiler.
Status Babel had been installed into Croquet as an experiment. Further exploration is required.
Priority Moderate to High
Gate Keeper Mark McCahill
JustInTimeCompilers
Description Croquet will need a Just In Time compiler for high performance simulations. This will certainly be some form of Jolt when it is ready for prime time.
Status Jolt (Pepsi/Coke) continues to evolve at a nice pace. It is not yet ready for inclusion inside of Croquet, but I think we should consider some experiments with the framework sooner rather than later.
Priority Low until critical mass of functionality - high after that. Beware of error 33.
Gate Keeper Andreas Raab
CroquetDocumentation
Description The documentation for Croquet needs to include a number of documents.
- User’s Guide – this document introduces new users to Croquet, how to set it up, how to connect to other users, and how to extend the system via a scripting engine.
- Programmer’s Guide – Describes how to develop Croquet applications, and gives an overview of the philosophy and architecture.
- Programmer’s Reference – The components of the architecture are described in detail, such that the programmer can easily reference functional capabilities by name or by description.
- Cookbook – Describes how to construct numerous example applications.
Status Preliminary documents have been developed, though these will have to be mostly rewritten.
Priority Critical
Gate Keeper David A. Smith
Demo Applications
Description Two types of Croquet demos need to be developed. A number of key demos that demonstrate how Croquet applications should be created, as well as several demonstrating new and interesting Croquet features.
Status The Croquet SDK currently contains demonstrations of how to create Master and Participant morphs. There are also demos of the University of Wisconsin's Kids First and a basic menu-driven environment using Tweak.
Priority Moderate
Gate Keeper Liz Wendland
ParallelObjectSupport
Description We presume that support of parallel architectures will be extremely interesting in the next few years.
Status Preliminary discussions.
Priority Low
Gate Keeper David A. Smith
Multi-core
Description TBD
Gate Keeper Andreas Raab
WicketDesignSystem
Description Wicket is the exemplar collaborative CAD application we are building in Croquet to test out the ideas of the interactors-tasks-replicants. Many of the ideas behind “Wicket” came from earlier work on Virtus Walkthrough, as well as more recent efforts inside of Croquet itself
Status Under low priority design.
Priority Low.
Gate Keeper David A. Smith
Library of Stuff/Content
Description Develop new open source content library.
Gate Keeper Philippe Van Nedervelde
