Medium 9781449309961

Building Web Applications with Erlang: Working with REST and Web Sockets on Yaws

Views: 1706
Ratings: (0)

Why choose Erlang for web applications? Discover the answer hands-on by building a simple web service with this book. If you’re an experienced web developer who knows basic Erlang, you’ll learn how to work with REST, dynamic content, web sockets, and concurrency through several examples. In the process, you’ll see first-hand that Erlang is ideal for building business-critical services.

Erlang was designed for fault-tolerant, non-stop telecom systems, and building applications with it requires a large set of skills. By the end of the book, you’ll have the information you need to build a basic web service and get it running.

  • Explore the power of Erlang and REST for building web services
  • Serve static and dynamic content with the Yaws web server
  • Use different methods for outputting data to user, such as encoding Erlang data structures into JSON or XML
  • Build an application to listen for HTTP requests, process them, store data, and return useful data
  • Go beyond the request-response model—push data to clients with web sockets
  • Use Erlang and Yaws to stream data from the server to a client

"A book which is truly needed and will help get Erlang to the next level."

—Francesco Cesarini, CEO of Erlang Solutions, author of Erlang Programming.

List price: $15.99

Your Price: $12.79

You Save: 20%

 

13 Slices

Format Buy Remix

1. Building Scalable Systems with Erlang and REST

ePub

In the early days of the Web, building systems was simple. Take a Linux box, put Perl or PHP on it, add Apache and MySQL, and you were ready to go. Of course, this system was pretty limited. If you wanted to scale it past one or two servers it got real hard, real fast. It turns out that building scalable distributed applications is difficult, and the tools available to build them are often less than ideal.

Over the first decade of the 21st century, companies like Google, Amazon, eBay, and many others found that they needed to scale not to a few servers but to a few thousand servers, or even tens or hundreds of thousands or more. This requires a very different way of thinking about how to build a system, and dropping many of the patterns that had been used in the past for smaller systems.

One alternate recipe that offers scalability, resilience, and flexibility is to create your sites and applications using Erlang, with the frontend being defined by a variety of web services.

 

2. Getting Started with Yaws

ePub

Most developers who are moving from other web development environments to Erlang and Yaws will have used other web servers such as Nginx or Apache. The Erlang Yaws web server performs the same basic tasks, but the details of performing common actions are often different.

Erlang is not only a language, but also a runtime system and something that looks a lot like an application server. As such, Erlang and Yaws (or other web servers) will fill the same role as Apache/PHP/MySQL and other components all in one system.

The major differences between Erlang/Yaws and Apache/PHP have a lot to do with how Erlang tends to set things up. Erlang assumes that systems will be clustered, and processes in Erlang are somewhat different from those used in many other systems.

If youve used Apache with mod_php, you may remember that each request is handled by a process or thread (depending on how things are set up). The classic Common Gateway Interface (CGI) would start a new process for every request. These threads and processes are constructions of the OS and are relatively heavyweight objects. In Erlang the processes are owned not by the OS, but by the language runtime.

 

3. Appmods: Dynamic Content in Yaws

ePub

When developing a web service, there are times when the developer does not want to map the URL that is sent by the web browser to an actual file. This may be because all data is stored in a database or is generated dynamically. In this case you want the browser to go to some URL and have the server create the content and return it.

Appmods are to Yaws what Apache modules are to Apachea way to create a custom handler for a URL in the web server. When using an appmod, the browser will send a standard URL, but instead of having a static file sent back to the user, Yaws will execute some Erlang code and send the response back to the user. This allows for fully dynamic content in the Yaws web server.

The difference is that in the case of an appmod, out/1 will need to create the full content of the response, while in a .yaws file it may just be a small part of it. In many cases, the response will not be HTML but will be some other format, such as XML (XML), JSON (JSON), or even something like an audio file, CSV text file, or PDF.

 

4. Implementing REST

ePub

So far this book has focused on the parts of how to build a web service, such as how to use a template and respond to a request. It is now time to expand our view to building larger, more complex integrated services to deal with complex business requirements.

In this chapter I will show how to build a simple RESTful service to store a list of airports. The service will use GET to request a list of airports, POST to add a new airport, PUT to update an airport, and DELETE to remove one.

This chapter will focus on the technical aspects of how to take an HTTP request in to some Erlang code and turn it into the correct output data or server actions. So, for example, the user may send an HTTP POST to create a new airport record, and the code will then store that data in a Mnesia database.

Most of what is in this chapter also applies to any form of web services insofar as they require the server to examine the incoming request and make an appropriate response.

When an HTTP request is made to the server there are a number of pieces of data that come with that request. All of these are sent to the out/1 function via the Arg data structure, and they can be extracted from that data structure. In some cases there are preexisting functions to extract the data, in others you will have to create functions to extract what you need.

 

5. File Upload

ePub

While being able to submit a form or other post variables via Ajax is a useful thing, sooner or later most applications will want to let a user upload a file, for example an avatar image or a video.

Yaws allows an application to receive a file from the user in the standard upload format that users of other server-side technologies such as PHP have come to expect.

When uploading a file in PHP, PHP buffers the input into the /tmp directory and then sends the finished file to your program. In Yaws, the server sends the file in chunks to your code. So instead of getting a complete file, the programmer has to be ready to get a stream of chunks of data. This does put a little more work onto the programmer, but it also allows the programmer to work with an incomplete upload if that is desired.

The examples in this chapter are taken from the example on the Yaws documentation website with a few minor changes. The full working code is in Example5-4, while the examples preceding it show parts of the code for explanation.

 

6. WebSockets

ePub

Traditionally HTTP is not very good for live communications. The communication stream is controlled by the client and is really designed for the case where the client wants to load or set data from time to time. A number of methods have been used to simulate TCP-socket-like behavior over HTTP but none of them works very well. HTML5 introduced the idea of WebSockets, a full-on, bi-directional communication channel between the browser and a server.

In some ways WebSockets take the opposite approach to dealing with interactions between the browser and the client than REST does. REST is built around the idea that the browser (or other client) will send a number of discrete requests to the server of the form, show this data, or perform some action.

As WebSockets are not supported in all browsers, having a cross-platform way of handling communication would be helpful. This can be done with the JavaScript package Socket.io (http://socket.io) and the Erlang package socket.io-erlang (https://github.com/yrashk/socket.io-erlang).

 

7. Streaming

ePub

Sometimes you want to stream data from a server to a client, for example, for an Internet radio station or a service like Pandora or Ustream. Yaws can do this quite well and with minimal effort on the part of the programmer.

The difference between streamed data and a standard HTTP connection is that a stream can remain open for a long period of time (oftentimes hours or days) and send data to the client for that entire time. However, unlike WebSockets (see Chapter6) a stream is a one-way data connection and will normally be binary data like music or video as opposed to textual data in a WebSocket.

To set up streaming in Yaws, the out/1 function should return the tuple {streamcontent, MimeType, FirstChunk} as in Example7-1.

Example7-1.Setting up streaming (stream.yaws)

You must also spawn a new process to actually send the data to the client. This is done in Example7-1 with the call to spawn/3. This will create a new process and pass the process ID of the creating process, as shown in Example7-2. When creating that process, the out/1 function passes its own PID via the self/0 function to the function streaming:stream_data/1.

 

8. Using the HTTP Client

ePub

Sometimes you need not to create an HTTP service but to consume oneperhaps to use a RESTful service or other API, or to load-test a server by hitting it with lots of requests at once. Erlang provides an HTTP client API that lets you do this. You can find the manual page for this module at http://www.erlang.org/doc/man/httpc.html. The httpc module is part of the standard Erlang distribution and does not have to be installed separately.

The first thing to do before using the httpc module is to start the inets service by calling inets:start() or application:start(inets). If you are running an application this can be done from the .erlang file or from the command line in testing. If you do not start inets, httpc will not work correctly. [12]

If there are some options that must be set for all calls then you can use the function httpc:set_options/1 or httpc:set_options/2. There are a number of options that can be set here, including all of the standard ones you would expect. Of particular note is the max_sessions option, which defaults to 2. In addition, if you need to set a proxy server you can do it here with the Proxy option. When calling set_option it will return ok or {error, Reason}.

 

9. Building an Application with OTP

ePub

So far this book has shown small pieces of Erlang or other code to demonstrate one idea or another. This chapter does something a bit different. Here I will develop a larger application to demonstrate how all the parts of an Erlang- and Yaws-based web application hang together.

This application will allow a bunch of users to notify each other of updates in status. Whenever a users status changes in their browser or on their phone, that change will be made available by the server. It will also keep track of each users status so when a user signs in, she can see all the existing status messages. This application can serve as the base of many distributed applications, and I hope it will prove illustrative of how to build an application in Erlang.

Feel free to use this module as a basis for your own product. If you do something really cool with it, please let me know!

This application will also split the application into layers: well have a server level that will coordinate between the users, and a web frontend that will use a simple web interface.

 

A. Installing Erlang and Yaws

ePub

Erlang can be installed from http://www.erlang.org. There are versions for Mac, Linux, and Windows. In addition, most popular Linux distributions have packages for Erlang. As of this writing, the current version is R15B01.

Many modern Linux distributions will feature Erlang and Yaws in their basic package system. So from Ubuntu, doing a simple sudo apt-get install yaws will install Erlang and Yaws (and any dependencies). Similarly, Fedora Linux includes both Erlang and Yaws as part of the basic packages, which can be installed via yum.

Yaws can be found at http://yaws.hyber.org and downloaded from there. Once again, most Linux distributions will include packages. As of January 2012, the current version is 1.92.

In Ubuntu Linux, doing a simple apt-get install yaws will install Yaws with all the dependencies, including Erlang itself.

For Microsoft Windows, there are installers for Yaws that can be downloaded from http://yaws.hyber.org. However, before Yaws can run, Erlang itself must be downloaded separately.

 

B. Beyond Yaws

ePub

This book has been an introduction to using Yaws with Erlang. However, there are a number of other web servers and frameworks for Erlang that may make more sense for a specific project.

In addition to Yaws, there are two other Erlang web servers in active development: Cowboy and MochiWeb. Each has a different set of pros and cons that should be considered.

A detailed comparison of different Erlang web server options could be a book in and of itself. However, a good place to start is http://www.ostinelli.net/a-comparison-between-misultin-mochiweb-cowboy-nodejs-and-tornadoweb/, which attempts to compare Misultin, MochiWeb, and Cowboy along with Node.js (JavaScript) and Tornadoweb (Python).

Cowboy (https://github.com/extend/cowboy) is a new web server for Erlang designed to be small, fast, and modular. It is also designed to be easy to embed in other applications, which can be useful if you are creating a web interface to a larger Erlang application.

 

C. Interfacing with Ruby and Python

ePub

So youve been reading this book and thinking that Erlang sounds amazingly cool. You already have a web infrastructure in PHP, Python, or Ruby but you would love to be able to build some structure in Erlang. There are several ways you can do this. You could use something like RabbitMQ to couple different parts of an application with queues. You could also create a web service in Erlang and access it from another language, or send data over a socket. However, what would be really nice is being able to have native communications between Erlang and some other language.

This is actually fairly simple to do. As we know by now, Erlang processes can communicate over a wire protocol. When you send a message from process A to process B, all the data is serialized in some way and sent across to the other process to receive. Knowing that, you might think it would be pretty easy to create a package in another language that can speak that protocol and work with it, and of course you would be right. There are packages for Ruby and Python (and others) that can do that quite well. (Sending Erlang functions to other languages probably wont work.)

 

D. Using Erlang with Emacs

ePub

For many Erlang developers, the editor of choice is Emacs. As a longtime Emacs user I have found there to be many reasons for this.

One feature of the Emacs Erlang mode is that you can compile a file by typing C-c C-k, which will open Erlang in a shell and compile the buffer you are in. You can also copy and paste code from a buffer to an Erlang shell.

Being able to copy and paste code from a buffer to the Erlang REPL makes it very easy to explore solutions to a problem. The programmer can create a simple module and load it into Erlang with C-c C-k, then create a bunch of test cases that are copied via standard Emacs editing to the shell to ensure that the code is working correctly. This does not replace formal testing, but supplements it when doing development.

In addition, the Erlang mode has a set of templates that can be used to create common structures. So if you need to work with the OTP gen_server pattern, you can generate the skeleton of that structure by opening up a new buffer and selecting the correct structure. The template for gen_server is shown in ExampleD-3 at the end of this chapter.

 

Details

Print Book
E-Books
Slices

Format name
ePub
Encrypted
No
Sku
9781449320652
Isbn
9781449320652
File size
0 Bytes
Printing
Not Allowed
Copying
Not Allowed
Read aloud
No
Format name
ePub
Encrypted
No
Printing
Allowed
Copying
Allowed
Read aloud
Allowed
Sku
In metadata
Isbn
In metadata
File size
In metadata