The 4th Dimension Web Application Platform INTRODUCTION 4th Dimension s combination of flexibility and simplicity provides organizations with the agility they need to deliver scalable and time-critical Web applications. Its integrated architecture combines an embedded RDBMS, Web server, and unified programming language making it possible to develop, test, and deploy applications in stand-alone, client-server, 3-tier, or widely distributed n-tier configurations with little or no reprogramming. In addition to this flexibility, there is no need to learn a monolithic framework or master a complex configuration management system. Of course, 4th Dimension is not a closed box. Solutions can also combine tools from other vendors, like an Oracle RDBMS or an Apache Web server. A variety of industry-standard connectivity tools and technologies, including ODBC, XML, FastCGI, and SOAP, enable 4th Dimension solutions to partner with virtually any system. This white paper examines in detail how these strengths and features of the 4th Dimension environment lend themselves to powerful Web application development. WEB APPLICATION SERVER PLATFORMS System architecture Web applications are commonly composed of three layers: the Web server, the application server, and the database server (Figure 1). These conceptual layers can be implemented as an integrated program on one machine or as distinct programs on different machines. Additional machines can be added at each layer for scalability, testing, and fault tolerance. In practice, a single Web application can run on anything from a single machine to hundreds of machines in a distributed environment. Selecting the appropriate design within this continuum is a challenge for project sponsors, management, and technical staff. System architectures are more difficult to change than system features so selecting the wrong architecture is a substantial project risk. Nobody wants to pick a tool or strategy that becomes a costly deadend when application requirements or loads expand. On the other hand, overbuying threatens budgets and deployment schedules. According to the Gartner Group (Research Note SPA-14-2716, August 2001), [T]here is a great disconnect between the appropriate and the actual in the application server deployment practices of mainstream enterprises. From 1998 through 2000, this disconnect collectively has cost enterprises more than $1 billion in overpayments for never-to-be-used capacity. From 2001 through 2003, it may cost them an additional $2 billion. Balancing simplicity and capacity Projects need to balance the risk of using a system that cannot grow to meet future demands against the risk of investing in a system that is too complex. More features and greater flexibility in deployment typically come at the cost of longer learning curves and higher maintenance costs. In contrast, simplicity in learning, developing, deploying, and maintaining applications typically comes at the cost of limited features and performance options. 4th Dimension is distinguished by a high degree of flexibility coupled with a low degree of complexity (Figure 2). Figure 1: The Three Layers of Web Applications HTTP://WWW.4D.COM/2003 1
The 4th Dimension Web Application Platform client-server application, or a widely distributed application. Because multi-user deployment is a core concept in the 4th Dimension environment, switching between single user and client-server is typically as easy as relaunching the program. Web applications share the same language, RDBMS, data, and business logic as other environments, reducing rework and eliminating inconsistencies. Figure 2: 4th Dimension couples a high degree of flexibility with a low degree of complexity. The 4th Dimension Web development environment delivers this unusual combination of strengths through a core architecture that can be either tightly integrated or entirely open (Figure 3). Flexibility: An open architecture 4th Dimension also supports Web application designs that combine multiple programs and services. Integration is essential in modern IT infrastructures as organizations commonly have multiple existing systems and development groups. Rather than writing and rewriting all corporate services on a single platform, the trend is increasingly to implement heterogeneous systems with homogenous communications infrastructures. Over the years, the standards for distributed and object-oriented component network architectures has matured. Today, 4th Dimension supports a wide-range of contemporary standards, including ODBC, a native HTTP server, FastCGI, XML, and SOAP. True to 4th Dimension s overall philosophy of providing flexibility without needless complexity, these tools can be deployed almost anywhere within the 4th Dimension environment (Table 1). Because these tools are each part of the native 4th Figure 3: The core architecture of the 4th Dimension Web development environment can be either tightly integrated or entirely open. Simplicity: An integrated architecture 4th Dimension s unified language can be applied to the RDBMS, Web server, Web services and traditional clientserver environments, as well as in client applications to large enterprise databases such as Oracle. This seamless integration simplifies application development and maintenance, lowering development time and lifetime costs. In fact, in a recent Ventera study 4th Dimension was the only Web application environment studied that earned a perfect score for development time (Table 6, page 9).The same 4th Dimension system can be deployed as a single-user, multi-process application, a Table 1: 4th Dimension Services and Configurations Dimension environment, adding them to a system requires no complicated or expensive configuration or rework. For example, adding a second Web server can be no more difficult than launching a second copy of 4D Client from a second machine. Matching the architecture to the project The 4th Dimension product family s support of both integrated and open architectural approaches offers development teams an unusual degree of freedom. In practice, the architecture can evolve with the project. This frees the development team to match the architecture to the project and the current project 2 HTTP://WWW.4D.COM/2003
phase. Consider some of the key advantages of each approach listed in Table 2. we ll now consider 4th Dimension s options for scaling and integration in more detail. Integrated Architecture Simplified development and testing Short development time Automatic connection pooling and caching Leverages one skill set for many tasks Fewer points of failure Simplified configuration management Open Architecture Highly scalable Readily faulttolerant Reuses existing resources and systems Minimizes disruption from one system component becoming unavailable SCALABILITY 4th Dimension s embedded tools lend themselves to scaling and load balancing. A Web application can initially be deployed entirely within a single-user copy of 4th Dimension with the integrated RDBMS, application server, and Web server. This configuration can typically handle the needs of a workgroup or small business without additional software. As demand on the system increases, the system can be moved to 4D Server to split the database from the application and Web server functions. This step typically requires zero recoding. Multiple copies of 4D Client can then act as application and Web servers to increase performance and to support load balancing. Using redundant copies of 4D Client is a particularly attractive option because it requires little or no additional programming, and adds very little to the ongoing costs of maintaining and managing the system. Finally, external systems can be integrated for the RDBMS, application server, and Web server layers as illustrated in Figure 4. Table 2: Advantages of Integrated and Open Architectures The advantages of a tightly integrated architecture and product family are well-matched to workgroup projects, vertical market applications, and virtually any project at its beginning. Complex requirements are made even more difficult to implement if the development tools themselves are complex. Development teams can apply 4th Dimension s well-known ease-of-use and RAD capabilities to requirements discovery, prototyping, load testing, and initial usability testing. As requirements stabilize and load increases, layers of the system can be scaled in a variety of ways to match actual current and readily anticipated needs. With this evolutionary architectural approach, a project can maintain the simplest sufficient system at all times. This agility provides a strategic advantage in today s business environment. In the words of one 4th Dimension customer: From day one PowerSchool has used 4th Dimension extensively, and we consider it a uniquely powerful tool for both development and deployment that has given us a tremendously huge competitive advantage in our market. Frankly, we ve pulled far ahead of our competitors on many fronts and we would not be where we are today had we used another tool set. - Greg Porter, Founder and Vice President of PowerSchool, Inc. Keeping evolutionary architectural approach in mind, Figure 4: 4th Dimension s embedded tools lend themselves to scaling and load balancing. 4th Dimension s wide variety of scaling solutions deliv- HTTP://WWW.4D.COM/2003 3
The 4th Dimension Web Application Platform Table 3: Scalability Options ers powerful scaling abilities within the integrated toolset. As an example, a single 4D Server system with four copies of 4D Client can handle millions of hits a day on inexpensive equipment. 4th Dimension s open architecture provides access to best-of-class products in the RDBMS, Web, and Web services categories. This feature is critical to organizations that need to integrate existing systems with the Web or that require near-instantaneous access to millions of rows of data. Table 3 lists some of the most common embedded and external options for scaling 4th Dimension solutions. All of the external scalability options depend on connectivity and integration with other systems. With the emergence of standards like SOAP, system interconnectivity is an ever more important component of system design and tool selection. Let s review what 4th Dimension has to offer in this important area. CONNECTIVITY AND INTEGRATION Overview of tools 4th Dimension was the first desktop environment to provide access to SQL databases with the 4D SQL Server product. Since then, 4th Dimension has continued to deliver the tools development groups need to bring programs together. Existing 4th Dimension solutions integrate with and support ColdFusion MX, Java, ASP,.NET, Perl, Python, PHP, WML/WAP, XML, IIS, Apache, 4D WebSTAR, and any ODBC database. Of chief importance to Web application developers are: the embedded 4D Web server 4D ODBC and 4D for OCI XML and SOAP FastCGI Native TCP/IP Each of these tools can be used directly within the core 4th Dimension development environment using the 4th Dimension language. The embedded 4th Dimension Web server 4th Dimension applications can embed a full HTTP server with just the click of a button. No separate external program is required, so configuration is simplified dramatically. This is a particularly valuable feature for developers creating vertical market and/or workgroup solutions. Apart from the convenience provided by an embedded Web server, there are some deeper benefits for any developer to consider. Dynamic Web sites that combine Web code and remote database data face fundamental performance challenges. The repeated opening and closing of database connections, and the repetitive retrieval of remote database rows are timeconsuming and resource intensive. The natural solution to these problems is to pool database connections and cache data. With the integrated 4th Dimension tools, these processes are handled automatically. Out of the box, the embedded Web server and application server inherit a continuous, multi-process connection to the embedded RDBMS. The Web application code can run numerous simultaneous requests against the database with no need for complicated programming. The central database automatically caches data intelligently for the benefit of all clients. No special coding or tuning is 4 HTTP://WWW.4D.COM/2003
required for developers to take advantage of this smart cache. 4D ODBC and 4D OCI With 4D ODBC, code running under 4th Dimension, 4D Client, or 4D Server can act as a client to any ODBC data source. 4D for OCI provides direct access to Oracle servers through the Oracle Connectivity Interface (OCI). The obvious uses for these tools are to incorporate existing data in a Web application or to move data to Oracle for greater performance. When 4th Dimension is used in this way, it effectively becomes a piece of middleware. In this role, the value of 4th Dimension s multiprocess support demonstrates itself. As discussed already, Web application servers and middleware often improve remote database access through database connection pooling. Using the 4th Dimension language, a 4th Dimension system can open a number of connections with a remote database and maintain them continuously. The developer has complete control over how many connections are opened, how long they remain open, and when they are closed down. As an example, here is a line of sample code that shows how a developer launches a method within its own process: $process_id:=new process("communicate_with_oracle"; Use_32K_of_RAM;"Oracle communication process.") Using the native RDBMS, caching data is equally simple and natural. Avoiding redundant network queries can reduce the load on the main RDBMS, reduce network traffic, and improve Web application responsiveness. Caching works well in a variety of applications, including systems that collect data for later posting, or that depend on largely static data like product descriptions. XML and SOAP Heterogeneous data systems are an inescapable fact of life in most organizations. Several distributed, object/component based architectures for integrating systems have emerged over the years to accommodate this state of affairs, including DCOM, CORBA, XML-RPC, and SOAP. The 4th Dimension environment supports SOAP natively both in the role of a client and as a Web service provider. The value of SOAP rests on a very simple idea: Any program should be able to communicate with any other program through a common description language. SOAP uses XML for the common description data, an ideal choice since XML is designed to handle all forms of data in a generalized manner. Instead of having to write specific parsers and formatters, the XML system acts as a universal parser. XML dramatically reduces the complexity and incompatibility problems that otherwise typically attend interchange formats. SOAP and XML very neatly reduce the complexity of connecting heterogeneous systems. Instead of writing connectivity drivers for specific program pairs (of which there are millions), each program or platform learns to read and write the common request language, thus opening each program to the entire universe of other programs that can communicate through SOAP. It should not matter if a program is written in COBOL and running on a 3270 or written in Java and running on a PDA. Apart from automatic support for SOAP message parsing and construction, the 4th Dimension language provides low-level commands for manipulating XML data directly. While SOAP is agnostic about which transport protocol is used to exchange messages, it is almost universally implemented through an HTTP interface. The 4th Dimension environment leverages its embedded Web server to integrate SOAP support seamlessly. FastCGI The FastCGI protocol was invented as an alternative to the performance limitations of CGI and constraints of vendor-specific Web APIs. The principle advantages of FastCGI are: Simplified porting applications between platforms. A vender-neutral open standard. More processor efficient than CGI calls. Designed to support distributed applications. Lightweight and stable. FastCGI is supported by all major Web servers, including Apache, IIS, and 4D WebSTAR either natively or through an additional module. The 4D FastCGI component enables 4th Dimension programs to listen and respond to FastCGI requests. Native TCP/IP New Internet standards emerge continuously. Rather than force developers to depend on 4th Dimension to add support for each standard in the core feature-set, 4th Dimension includes a language extension called 4D Internet Commands. Using these commands, developers can add any TCP/IP-based service they require. The HTTP://WWW.4D.COM/2003 5
The 4th Dimension Web Application Platform Table 4: Features Supporting System Robustness toolkit includes simple, high-level commands for common standards, including DNS lookups, FTP, SMTP, POP3, SSL, and IMAP. Developers have used the low-level TCP/IP connectivity commands to create custom services, implement other standards, and build Web clients and servers. 4th Dimension-to-4th Dimension communication 4th Dimension can act as a client or a server for FastCGI, Web, SOAP, and ODBC requests (Table 1, page 2). Consequently, 4th Dimension applications can communicate amongst themselves using all of the tools outlined here. In addition, 4th Dimension includes a custom database API called 4D Open for applications that need to embed 4th Dimension client behavior in other applications. 4D Open is commonly used to create 4D-Serverto-4D-Server systems. Leveraging existing resources A key advantage of integration using any of the tools discussed above is the ability to leverage existing resources, including developer skills, tools, data, and services. Rather than rewriting entire systems in 4th Dimension, developers can implement new services and functionality in 4th Dimension that access and reuse existing resources. Another approach is to re-implement functionality in 4th Dimension that requires conversion in any case. This strategy should be of particular interest to organizations migrating code from Microsoft DNA to.net. A recent Gartner Group study (Research Note COM- 16-1385, June 2002) estimates that: The cost of converting from Windows DNA to.net may surprise even those who are familiar with these technologies...expenses can be as much as 60 percent of the original development cost...the cost of migration is, at best, 40 percent of the original development cost...most organizations will not experience the best case scenario. 4th Dimension SOAP enables organizations to implement Web services quickly and easily without disrupting existing systems. Because SOAP is standards-based, the underlying differences in implementation language 4D, Java,.NET become irrelevant to the overall system. ROBUSTNESS Robust systems handle the unexpected gracefully. 4th Dimension s integrated backup, scaling, and connectivity options provide tools to create and maintain highly robust and fault-tolerant systems. Table 4 relates these features to the layers of a typical Web application. Security Security is a core requirement of both client-server and Web applications. Few events are less welcome than the loss of confidential data or the negative publicity surrounding security breaches. The 4th Dimension environment provides a variety of tools to secure and protect data within the database, the local network, and over the Internet. These features are strong enough that the Army selected 4D WebSTAR as its Web server for the <http://www.army.mil> site after a well-publicized attack on its previous server solution. Table 5 on the next page summarizes 4th Dimension s wealth of security features. THE 4TH DIMENSION WEB DEVELOPMENT PARADIGM Separating data, content, and logic Web system development is a multi-disciplinary job, potentially involving programmers, writers, graphic designers, and usability engineers. No one part of the team should become a bottleneck for the others. The 4th Dimension environment supports a neat partitioning of data, presentation logic, business logic, and graphic design, which brings substantial efficiencies both during initial system development and during ongoing program development and maintenance. For example, instead of the graphics team waiting on the programming team, all teams can advance their tasks 6 HTTP://WWW.4D.COM/2003
Table 5: 4th Dimension Environment Security Features simultaneously. To fully appreciate how 4th Dimension offers this economy of purpose, we first need to review the major Web application strategies in the marketplace today. Scripts and tags There are two widely-used approaches to building dynamic Web systems: scripts and tags. The scripting approach is exemplified by Microsoft s ASP and.net technologies, while tag-based systems are well represented by Macromedia s ColdFusion MX platform. Each approach has strengths and weaknesses worth considering. Unusual amongst Web platforms, 4th Dimension supports both a script-based architecture and a tagbased architecture. In either case, 4th Dimension s machine-language compiler and embedded database offer strategic advantages for performance and configuration management. Script-based systems Scripting architectures enable developers to embed programming instructions directly within HTML pages that are then pre-processed by the Web application server on demand. This approach gives developers tremendous flexibility and greatly simplifies testing individual scripts. Unfortunately, embedding scripting can make system updates, configuration management, testing, and some aspects of system security costly or difficult. Also, since the scripting is combined with the actual HTML, there is often a poor division of program logic, presentation logic, and graphic design. Situations can arise where the programming group becomes a bottleneck for the rest of the project team. Additionally, another practical disadvantage of typical scripting solutions is that they are complicated to move between machines or to update. Configuration management can become increasingly burdensome over the lifetime of even a moderately complex project. Script-based systems also tend to run as interpreted or byte-code executed on demand. This approach is significantly slower than running compiled code, increases Web response times, and needlessly taxes the Web application server. 4D Compiler 4D Compiler, a product included in the 4th Dimension Development suite, is a native compiler for 4th Dimension projects. As we ll see, 4D Compiler offers great advantages in either script or tag-based Web environments. Compilation can speed up text handling operations, like parsing and building text blocks, hundreds to thousands of times. Optimization of this magnitude is of particular benefit to Web applications because the overwhelming majority of Web code involves processing text. 4D Compiler is a true machinelanguage compiler. The code produced is not tokenized code or byte-code like Java. The compiled system can be targeted to Windows, Mac OS, OS X or all of the above, and no special coding is required to support these different platforms from a single 4th Dimension source code project. Scripting in 4th Dimension Scripting in 4th Dimension is by the Active4D product available from www.aparajitaworld.com. With Active4D, developers can combine HTML and code in source pages, much as they would using ASP. In addition, session support and other advanced features are automatically included in the project. The combination of Active4D and the native 4th Dimension environment offers several advantages over many scripting environments: HTTP://WWW.4D.COM/2003 7
The 4th Dimension Web Application Platform Code can be embedded in Web pages, external libraries, or methods within the host database application. Libraries promote reuse and cleanly partition HTML from business logic. Moving code into 4th Dimension methods offers security and the speed benefits of a machine-language compiler. Code and dynamic page templates can be stored as individual disk files or embedded within the database. Entire virtual structures can be maintained without any physical disk files outside the database, if the developer wishes. Embedding the pages within the database eliminates a host of configuration management challenges, simplifies security, automates duplicating the application server and core Web pages on multiple machines, and enables the developer to create high-performance caches. 4th Dimension s approach to scripting is truly best of class. Tag-based systems Tag-based systems naturally promote a clear separation of program code and HTML. There is no need for writing complete programs in HTML because high-level tags are embedded to manipulate database objects, invoke business rules, and perform routine presentation preparation tasks. This approach establishes a clear separation of concerns between programming and design that can increase overall project efficiency. A common limitation of tag-based systems, however, is that the tag collection is fixed by the vendor. Developers are limited by what the developer provides or, in some cases, what extensions they can develop in C++ or Java. In the case of 4th Dimension, the native tag set is quite small. The tags offer a fine balance between simplicity and power. Rather than implement a large but restrictive set of tags, 4th Dimension provides a handful of tags for the core tasks of dynamic Web systems: Extracting and formatting database data. Looping through data or based on logic. Conditional branching to support (e.g. changing the color or format of text based on logical tests or database values). Invoking complex behavior, specialized code, and business logic through method callbacks. Method callbacks free the developer to effectively extend the tag system without limits and without using a low-level language like C++. The full resources of the 4th Dimension language and environment are available, including access to local data, remote data, and remote Web servers and services. Vertical market developers commonly use the 4th Dimension tagging system to create custom APIs for their products. Their customers can then customize their installations using a rich set of tags provided by the development organization without ever having direct access to the internal source code, and without having to learn the 4th Dimension language. This flexible and extensible approach is arguably one of the most unique in the marketplace today. Advantages of an integrated environment Both the script-oriented and tag-oriented strategies outlined previously take advantage of 4th Dimension s integrated tools. Since the database is local and embedded, pages, scripts, and actual user data can be accessed transparently. There is no extra effort, expense, or resource allocation required for database access the database is just there to access easily and efficiently. Likewise, the entire 4th Dimension language is available through callbacks to project methods. Therefore, the same business logic can be applied by all scripts and clients. 4th Dimension supports triggers to centrally and automatically enforce key database rules. Because of 4D Server s unique push code distribution system, business logic needs to be updated once on the 4D Server machine for it to be inherited automatically by each copy of 4D Client. This process is as close to a zero-maintenance server as is available in the marketplace today. Integration with popular Web design tools True to its philosophy of separating programming from Web design, 4th Dimension offers free extensions to Adobe GoLive and Macromedia Dreamweaver to support the native tagging system. These extensions enable Web designers to include 4th Dimension elements like table data formatting rules and controls in their designs with simple drag-and-drop. Designers do not need to learn or understand the 4th Dimension language to incorporate 4th Dimension content in their systems, so they are free to use the tools they like best to do the job they do best. 4th Dimension, of course, also supports Cascading Style Sheets (CSS), one of the most potent ways of simultaneously increasing Web design flexibility, system simplicity, and system performance. FEATURES OF THE 4TH DIMENSION DEVELOPMENT LIFECYCLE 4th Dimension and configuration management Organizations new to 4th Dimension are almost univer- 8 HTTP://WWW.4D.COM/2003
sally surprised at how simple it is to update custom programs. In all 4th Dimension environments, the program code and the database data are kept in distinct files. Therefore, a development team can distribute a new structure including routines, custom Web APIs, and other resources to one or more installation sites without installer scripts, data export, or other special work. These are the required steps for each site to install a typical update: 1. Quit 4th Dimension or 4D Server. 2. Replace the existing application file with the updated version. 3. Relaunch 4th Dimension or 4D Server. The update is complete. Under 4D Server, code and other resources are stored centrally on the server machine. As stored procedures, client machines, and processes including Web application servers use application objects, resources are copied and cached locally. This push distribution system avoids the necessity of manually configuring individual clients with each update, reduces network traffic, and increases performance through local caching. As noted earlier, the embedded database can even be used to store Web pages and scripts, so it possible to deploy a system simply by dragging-and-dropping two files onto a hard drive and launching 4th Dimension or 4D Server. Costs Having the best architecture and toolset is immaterial if the costs to develop and maintain the application far outweigh the business benefits. A recent independent study by Ventera, summarized in Table 6, shows that 4D had the fastest implementation time of all the products tested. In fact, it was the only product reviewed to receive a perfect score in this critical category. According to the Ventera researcher quoted in the study: The 4D Product Line provides end-to-end software including Database, Web Server, Application Server, and Integrated Development Environment (IDE). You can produce an entire Web site without purchasing any additional software. I also found the ability to change from a Single- User application to a Client-Server application and from a Client-Server application to a Web-based application very interesting. I cannot think of any other tool that enables a developer to do this. The simple, flexible deployment options admired by the Ventera researcher account for 4th Dimension s high marks in the study. Instead of mastering a monolithic framework or building for maximum capacity immediately, programmers can focus on maintaining the simplest sufficient system at every stage. The time and costs savings from this approach are self-evident. Indeed, initial development costs count for nothing if the product is difficult or costly to maintain. A recent Table 6: Ventera Study HTTP://WWW.4D.COM/2003 9
The 4th Dimension Web Application Platform be avoided because projects can start at the smallest appropriate size, develop quickly, and grow to meet genuine need as it arises. Table 7: Aberdeen Group 5-year VCO Study Findings Aberdeen Group study, summarized in Table 7, shows that solutions deployed with 4D s Web architecture cost about 35% of the industry average throughout the lifetime of the project. Stated another way, an organization can deploy three 4D-based solutions over the next five years for the price of a single typical solution, based on industry averages. CONCLUSION 4th Dimension s Web application architecture was designed expressly to maximize the benefits of rapid development while simultaneously avoiding the hazards of complexity and overbuilding. 4th Dimension s rapid development features allow organizations to deliver and maintain systems at roughly 70% less than the industry average. Standards-based connectivity and integration technologies amplify 4th Dimension s abilities by supporting widely distributed architectures and best of class programs in each application layer. All of these features contribute to 4th Dimension as an agile, powerful, and remarkably adaptable product family that deserves consideration for your Web application projects. The Gartner Group finding cited earlier bears repeating: From 1998 through 2000, this disconnect collectively has cost enterprises more than $1 billion in overpayments for never-to-be-used capacity. From 2001 through 2003, it may cost them an additional $2 billion. With the 4D Web application approach, this hazard can 10 HTTP://WWW.4D.COM/2003