Direct UI is the notion of treating a document window as more than just a WYSIWYG display. This includes treating the window as a control surface, inserting various markup, treating document elements as controls, and applying transforms/animations that allow the elements to momentarily transcend their document layout. It’s about making the document come alive.
It’s something that I thought heavily about in building my next document-based application. It’s also one of the reasons why I opted out of using any of Microsoft’s text-editing solutions, because of poor layout extensibility. Avalon text may support render transforms and animations of text, but those options are still too limiting. My NStatic tool also includes some direct UI, which has yet to be revealed, in order to present a more effortless user experience.
In Office, we see documents as a limited control surface with form controls, smart tags, Excel’s drag handle and Office 2007’s floatie. Visual Studio offers a variety of Intellisense controls. Office also includes markup for spelling errors, comments, and change tracking. With the possible exception of PivotTables and Table structures in Excel, document elements, however, are still bound to their layout and don’t typically exhibit control-like behavior. Such a feature typically requires a good understanding of the underlying document.
Refactor Pro probably makes more extensive use of direct UI than any other non-image-processing application. It includes extensive use of markup, in-document controls, and animations.
Designed for refactoring, Refactor Pro needs to acquire a fuller understanding of the user’s documents in order to relieve the user of tedious brain work. This also describes the smart applications that I am building. As applications become smarter, they will make greater use of markup and in-document controls.
Why do controls have to be alongside documents, you ask? It’s about moving the operation closer to the operand, eliminating the multi-step process of selection and command searching, interacting in an instantaneous and anticipative way.
I have also written posts about 3D user interfaces as well as a 3D framework that I built on top of GDI+. In addition to increasing realism, 3D interfaces also serve a major function. As more controls accompany documents over time, three-dimensional graphics will allow these controls to produce less clutter as rotation and perspective transformations tend to reduce the amount of screen real estate used.