Outside Connectivity II (Asterisk)
Storage Backends
The storage of messages on traditional voicemail systems has always tended to be overly complicated. Asterisk, on the other hand, not only provides you with a simple, logical, filesystem-based storage mechanism, but also offers a few extra message storage options.
Linux Filesystem
By default, Asterisk stores voice messages in the spool folder, at /var/spool/asterisk/voicemail/ <context> / <mailbox> . The messages can be stored in multiple formats (such as WAV and GSM), depending on what you specified as the format in the [general] section of your voicemail.conf file. Your greetings are also stored in this folder.
Here’s an example of what might be in a mailbox folder. This mailbox has no new messages in the INBOX, has two saved messages in the Old folder, and has busy, unavailable and name (greet) greetings recorded (as shown in Figure).
Figure- Sample mailbox folder
“IMPORTANT POINT– For each message, there is a matching msg####.txt file, which contains the envelope information for the message. The msg####.txt file is also critically important for message waiting indication (MWI), as this is the file that Asterisk looks for in the INBOX to determine whether the message light for a user should be on or off.“
ODBC Voicemail Message Storage
Asterisk enables you to store voicemail inside the database using the ODBC connector. This is useful in a clustered environment where you want to abstract the voicemail data from the local system so that multiple Asterisk boxes have access to the same data. Of course, you have to take into consideration that you are centralizing a part of Asterisk, and you need to act to protect that data, such as making regular backups and possibly clustering the database backend using replication.
Asterisk stores each voicemail message inside a Binary Large OBject (BLOB). When retrieving the data, it pulls the information out of the BLOB and temporarily stores it on the hard drive while it is being played back to the user. Asterisk then removes the
BLOB and the record from the database when the user deletes the voicemail. Many databases, such as MySQL, contain native support for BLOBs, but as you’ll see, with PostgreSQL a couple of extra steps are required to utilize this functionality. After com‐
pleting this section you’ll be able to record, play back, and delete voicemail data from the database just as if it were stored on the local hard drive.
Compiling the app_voicemail Module to Support ODBC Storage
In order to support writing voice messages to an ODBC database, the capability to do so must be compiled into the voicemail module.
Navigate to the directory where you downloaded your Asterisk source code. you should find it somewhere around here:
cd ~/src/asterisk-complete/asterisk/11
Run make with the menuselect argument:
$ sudo make menuselect
This will start the Asterisk Module and Build Selection interface. You will want to navigate to Voicemail Build Options and select ODBC_STORAGE :
Asterisk Module and Build Option Selection
Voicemail Build Options
— core —
( ) FILE_STORAGE
[*] ODBC_STORAGE
XXX IMAP_STORAGE
Then simply save, and run:
$ make install
and restart Asterisk. Your voicemail is ready to write to the database!
Creating the Large Object Type for PostgreSQL
While MySQL has a BLOB (Binary Large OBject) type, we have to tell PostgreSQL how to handle large objects. This includes creating a trigger to clean up the data when we delete from the database a record that references a large object.
Connect to the database as the asterisk user from the console:
$ psql -h localhost -U asterisk asterisk
Password:
At the PostgreSQL console, run the following script to create the large object type:
CREATE FUNCTION loin (cstring) RETURNS lo AS ‘oidin’ LANGUAGE internal
IMMUTABLE STRICT;
CREATE FUNCTION loout (lo) RETURNS cstring AS ‘oidout’ LANGUAGE internal
IMMUTABLE STRICT;
CREATE FUNCTION lorecv (internal) RETURNS lo AS ‘oidrecv’ LANGUAGE internal
IMMUTABLE STRICT;
CREATE FUNCTION losend (lo) RETURNS bytea AS ‘oidrecv’ LANGUAGE internal
IMMUTABLE STRICT;
CREATE TYPE lo ( INPUT = loin, OUTPUT = loout, RECEIVE = lorecv, SEND = losend,
INTERNALLENGTH = 4, PASSEDBYVALUE );
CREATE CAST (lo AS oid) WITHOUT FUNCTION AS IMPLICIT;
CREATE CAST (oid AS lo) WITHOUT FUNCTION AS IMPLICIT;
We’ll be making use of the PostgreSQL procedural language called pgSQL/PL to create a function. This function will be called from a trigger that gets executed whenever we modify or delete a record in the table used to store voicemail messages. This is so the
data is cleaned up and not left as an orphan in the database:
CREATE FUNCTION vm_lo_cleanup() RETURNS “trigger”
AS $$
declare
msgcount INTEGER;
begin
— raise notice ‘Starting lo_cleanup function for large object with oid
%’,old.recording;
— If it is an update action but the BLOB (lo) field was not changed,
don’t do anything
if (TG_OP = ‘UPDATE’) then
if ((old.recording = new.recording) or (old.recording is NULL)) then
raise notice ‘Not cleaning up the large object table,
as recording has not changed’;
return new;
end if;
end if;
if (old.recording IS NOT NULL) then
SELECT INTO msgcount COUNT(*) AS COUNT FROM voicemessages WHERE recording
= old.recording;
if (msgcount > 0) then
raise notice ‘Not deleting record from the large object table, as object
is still referenced’;
return new;
else
perform lo_unlink(old.recording);
if found then
raise notice ‘Cleaning up the large object table’;
return new;
else
raise exception ‘Failed to clean up the large object table’;
return old;
end if;
end if;
else
raise notice ‘No need to clean up the large object table,
no recording on old row’;
return new;
end if;
end$$
LANGUAGE plpgsql;
We’re going to create a table called voicemessages where the voicemail information
will be stored:
CREATE TABLE voicemessages
(
uniqueid serial PRIMARY KEY,
msgnum int4,
dir varchar(80),
context varchar(80),
macrocontext varchar(80),
callerid varchar(40),
origtime varchar(40),
duration varchar(20),
mailboxuser varchar(80),
mailboxcontext varchar(80),
recording lo,
label varchar(30),
“read” bool DEFAULT false,
flag varchar(10)
);
And now we need to associate a trigger with our newly created table in order to perform cleanup whenever we change or delete a record in the voicemessages table:
CREATE TRIGGER vm_cleanup AFTER DELETE OR UPDATE ON voicemessages FOR EACH ROW
EXECUTE PROCEDURE vm_lo_cleanup();
ODBC Voicemail Storage Table Layout
We’ll be utilizing the voicemessages table for storing our voicemail information in an ODBC-connected database. Table describes the table configuration for ODBC voicemail storage. If you’re using a PostgreSQL database, the table definition and large object support were configured in the preceding section.
Table of ODBC voicemail storage table layout
Here is an example of how to create this table under MySQL:
CREATE TABLE voicemessages
(
uniqueid serial PRIMARY KEY,
msgnum int(4),
dir varchar(80),
context varchar(80),
macrocontext varchar(80),
callerid varchar(40),
origtime varchar(40),
duration varchar(20),
mailboxuser varchar(80),
mailboxcontext varchar(80),
recording blob,
label varchar(30),
`read` bool DEFAULT false,
flag varchar(10)
);
A PostgreSQL example is in the previous section.
Configuring voicemail.conf for ODBC Storage
There isn’t much to add to the voicemail.conf file to enable the ODBC voicemail storage. In fact, it’s only three lines! Normally, you probably have multiple format types defined in the [general] section of voicemail.conf, but we need to set this to a single format
because we can only save one file (format) to the database. The WAV49 format is a compressed WAV file format that should be playable on both Linux and Microsoft Windows desktops.
The odbcstorage option points at the name you defined in the res_odbc.conf file (if you’ve been following along in this chapter, then we called it asterisk). The odbctable option refers to the table where voicemail information should be stored. In the examples
in this chapter we use the table named voicemessages .
Edit the [general] section of your voicemail.conf file so that the following values are
set:
[general]
format=wav49
odbcstorage=asterisk
odbctable=voicemessages
To create the users you can either separate voicemail context, or simply use the default voicemail section. Alternatively, you can skip creating a new user and use an existing user, such as 0000FFFF0001 . We’ll define the mailbox in the default section of the voice mail.conf file like so:
[default]
1000 => 1000,J.P. Wiser
Now connect to your Asterisk console and unload, then load the app_voicemail.so module:
*CLI> module unload app_voicemail.so
== Unregistered application ‘VoiceMail’
== Unregistered application ‘VoiceMailMain’
== Unregistered application ‘MailboxExists’
== Unregistered application ‘VMAuthenticate’
*CLI> module load app_voicemail.so
Loaded /usr/lib/asterisk/modules/app_voicemail.so =>
(Comedian Mail (Voicemail System))
== Registered application ‘VoiceMail’
== Registered application ‘VoiceMailMain’
== Registered application ‘MailboxExists’
== Registered application ‘VMAuthenticate’
== Parsing ‘/etc/asterisk/voicemail.conf’: Found
Then verify that your new mailbox loaded successfully:
*CLI> voicemail show users for default
context Mbox User zone newmsg
default 1000 J.P. Wiser
Testing ODBC Voice Message Storage
Let’s create some simple dialplan logic to leave and retrieve some voicemail from our test voicemail box. You can add the simple dialplan logic that follows to your extensions.conf file.
[odbc_vm_test]
exten => 100,1,VoiceMail(1000@default) ; leave a voicemail
exten => 200,1,VoiceMailMain(1000@default) ; retrieve a voicemail
Once you’ve updated your extensions.conf file, be sure to reload the dialplan:
*CLI> dialplan reload
Then configure your phone or client with the username odbc_test_user and password <supersecret>, and place a call to extension 100 to leave a voicemail. If successful, you should see something like:
— Executing VoiceMail(“SIP/odbc_test_user-10228cac”, “1000@default”) in new
stack
— Playing ‘vm-intro’ (language ‘en’)
— Playing ‘beep’ (language ‘en’)
— Recording the message
— x=0, open writing: /var/spool/asterisk/voicemail/default/1000/tmp/dlZunm
format: wav49, 0x101f6534
— User ended message by pressing #
— Playing ‘auth-thankyou’ (language ‘en’)
== Parsing ‘/var/spool/asterisk/voicemail/default/1000/INBOX/msg0000.txt’: Found
Now that you’ve confirmed everything was stored in the database correctly, you can try listening to it via the VoiceMailMain() application by dialing extension 200 :
*CLI>
— Executing VoiceMailMain(“SIP/odbc_test_user-10228cac”,
“1000@default”) in new stack
— Playing ‘vm-password’ (language ‘en’)
— Playing ‘vm-youhave’ (language ‘en’)
— Playing ‘digits/1’ (language ‘en’)
— Playing ‘vm-INBOX’ (language ‘en’)
— Playing ‘vm-message’ (language ‘en’)
— Playing ‘vm-onefor’ (language ‘en’)
— Playing ‘vm-INBOX’ (language ‘en’)
— Playing ‘vm-messages’ (language ‘en’)
— Playing ‘vm-opts’ (language ‘en’)
— Playing ‘vm-first’ (language ‘en’)
— Playing ‘vm-message’ (language ‘en’)
== Parsing ‘/var/spool/asterisk/voicemail/default/1000/INBOX/msg0000.txt’: Found
Verifying binary data stored in PostgreSQL
To make sure the recording really did make it into the database, use the psql application:
$ psql -h localhost -U asterisk asterisk
Password:
Next, run a SELECT statement to verify that you have some data in the voicemessages
table:
localhost=# SELECT uniqueid,dir,callerid,mailboxcontext,
recording FROM voicemessages;
uniqueid | dir | callerid
———+————————————————–+————–
1 | /var/spool/asterisk/voicemail/default/1000/INBOX | +18005551212
| mailboxcontext | recording |
+—————-+———–+
| default | 47395
|(1 row)
If the recording was placed in the database, you should get a row back. You’ll notice that the recording column contains a number (which will most certainly be different from what is listed here), which is really the object ID of the large object stored in a system
table. You can verify that the large object exists in this system table with the lo_list command:
localhost=# \lo_list
Large objects
ID | Description
——-+————-
47395 |
(1 row)
What you’re verifying is that the object ID in the voicemessages table matches what is listed in the large object system table. You can also pull the data out of the database and store it to the hard drive:
localhost=# \lo_export 47395 /tmp/voicemail-47395.wav
lo_export
Then verify the audio with your favorite audio application, such as play:
$ play /tmp/voicemail-47395.wav
Input Filename : /tmp/voicemail-47395.wav
Sample Size : 8-bit
Sample Encoding : wav
Channels : 1
Sample Rate : 8000
Time: 00:06.22 [00:00.00] of 00:00.00 ( 0.0%) Output Buffer: 298.36k Done.
Verifying binary data stored in MySQL
To verify that your data is being written correctly, you can use the mysql application to log into your database and export the voicemail recording to a file:
$ mysql -u asterisk -p asterisk
Enter password:
Once logged into the database, you can use a SELECT statement to dump the contents of the recording to a file. First, though, make sure you have at least a single recording in your voicemessages table: 20
mysql> SELECT uniqueid, msgnum, callerid, mailboxuser, mailboxcontext, `read`
-> FROM voicemessages;
+———-+——–+——————————+————-
| uniqueid | msgnum | callerid | mailboxuser
+———-+——–+——————————+————-
|1 | 0 | “Leif Madsen” <100>| 100
|2 | 1 | “Leif Madsen” <100>| 100
|3 | 2 | “Leif Madsen” <100> | 100
|5 | 0 | “Julie Bryant” <12565551111> | 100
+———-+——–+——————————+————-
+—————-+——+
| mailboxcontext | read |
+—————-+——+
| shifteight.org | 0 |
| shifteight.org | 0 |
| shifteight.org | 0 |
| default | 0 |
+—————-+——+
Having verified that you have data in your voicemessages table, you can export one of the recordings and play it back from the console:
mysql> SELECT recording FROM voicemessages WHERE uniqueid = ‘5’
-> DUMPFILE ‘/tmp/voicemail_recording.wav’;
Now exit the MySQL console, and use the play application from the console (assuming you have speakers and a sound card configured on your Asterisk system, which you might if you are going to use it for overhead paging), or copy the file to another system and listen to it there:
$ play /tmp/voicemail_recording.wav
voicemail_recording.wav:
File Size: 7.28 k Bit Rate: 13.1k
Encoding: GSM
Channels: 1 @ 16-bit
Samplerate: 8000Hz
Replaygain: off
Duration: 00:00:04.44
In:100% 00:00:04.44 [00:00:00.00] Out:35.5k [ | ] Hd:4.4 Clip:0 Done.
IMAP
Many people would prefer to manage their voicemail as part of their email. This has been called unified messaging by the telecom industry, and its implementation has traditionally been expensive and complex. Asterisk allows for a fairly simple integration between voicemail and email, either through its built-in voicemail-to-email handler, or through a relationship with an IMAP server.
Using Asterisk as a Standalone Voicemail Server
In a traditional telecom environment, the voicemail server was typically a standalone unit (provided either as a separate server altogether, or as an add-in card to the system). Very few PBXs had fully integrated voicemail (in the sense that voicemail was an integral part of the PBX rather than a peripheral device). Asterisk is quite capable of serving as a standalone voicemail system. The two most common reasons one might want to do this are:
1. If you are building a large, centralized system and have several servers each providing a specific function (proxy server, media gateway, voicemail, conferencing, etc.)
2. If you wish to replace the voicemail system on a traditional PBX with an Asterisk voicemail Asterisk can serve in either of these roles.
Integrating Asterisk into a SIP Environment as a Standalone Voicemail Server
If you want to have Asterisk act as a dedicated voicemail server (i.e., with no sets registered to it and no other types of calls passing through it), the process from the dialplan perspective is quite simple. Getting message waiting to work can be a bit more difficult, though.
Let’s start with a quick diagram. Figure shows an overly simplified example of a typical SIP enterprise environment. We don’t even have an Asterisk server in there (other than for the voicemail), in order to give you a generic representation of how Asterisk could serve as a standalone voicemail server in an otherwise non-Asterisk environment.
Unfortunately, Asterisk cannot send message notifications to an endpoint if it doesn’t know where that endpoint is. In a typical Asterisk system, where set registration and voicemail are handled on the same machine, this is never a problem, since Asterisk knows where the sets are. But in an environment where the sets are not registered to Asterisk, this can become a complex problem.
There are several solutions on the Internet that recommend using the externnotify option in voicemail.conf, triggering an external script whenever a message is left in a mailbox (or deleted). While we can’t say that’s a bad approach, we find it a bit kludgy, and it requires the administrator to understand how to write an external script or program to handle the actual passing of the message.
Figure-Simplified SIP enterprise environment
Instead you can statically define an entry for each mailbox in the voicemail server’s sip.conf file, indicating where the message notifications are to be sent. Rather than defining the address of each endpoint, however, you can have the voicemail server send
all messages to the proxy, which will handle the relay of the message notifications to the appropriate endpoints. The voicemail server still needs to know about the SIP endpoints, even though the devices are not registered directly to it. This can be done either through a sip.conf file that identifies each SIP endpoint, or through a static realtime database that does the same thing. Whether you use sip.conf or the Asterisk Realtime Architecture (ARA), each endpoint will require an entry similar to this:
[messagewaiting](!)
; a template to handle the settings common
; to all mailboxes
type=peer
subscribecontext=voicemailbox ; the dialplan context on the voicemail server
context=voicemailbox ; the dialplan context on the voicemail server
host=192.168.1. ; ip address of presence server
[0000FFFF0001](messagewaiting) ; This will need to match the subscriber
; name on the proxy
mailbox=0000FFFF0001@DIR1 ; Must be in the form mailbox@mailboxcontext
defaultuser=0000FFFF0001 ; this will need to match the subscriber
; name on the proxy
You will not want to implement this unless you have prototyped the basic operation of the solution. Although we all agree that SIP is a protocol, not everyone agrees as to the correct way to implement the protocol. As a result, there are many interoperability
challenges that need to be addressed in a solution like this. We have provided a basic introduction to this concept in this book, but the implementation details will depend on other factors external to Asterisk, such as the capabilities of the proxy. The fact that no device has to register with Asterisk will significantly reduce the load on the Asterisk server, and as a result this design should allow for a voicemail server that can support several thousand subscribers.
Dialplan requirements
The dialplan of the voicemail server can be fairly simple. Two needs must be satisfied:
1. Receive incoming calls and direct them to the appropriate mailbox
2. Handle incoming calls from users wishing to check their messages
The system that is passing calls to the voicemail server should set some SIP headers in order to pass additional information to the voicemail server. Typically, this information would include the mailbox/username that is relevant to the call. In our example, we are going to set the headers X-Voicemail-Mailbox and X-Voicemail-Context , which will contain information we wish to pass to the voicemail server.
sip.conf requirements
In the sip.conf file on the voicemail server, not only are entries required for all the mailboxes for message-waiting notification, but some sort of entry is required to define the connection between the voicemail server and the rest of the SIP environment:
[VOICEMAILTRUNK]
type=peer
defaultuser=voicemail
fromuser=voicemail
secret=s0m3th1ngs3cur3
canreinvite=no
host=<address of proxy/registrar server>
disallow=all
allow=ulaw
dtmfmode=rfc2833
context=voicemailbox
The other end of the connection (probably your proxy server) must be configured to pass voicemail connections to the voicemail server.
Running Asterisk as a standalone voicemail server requires some knowledge of clustering and integration, but you can’t beat the price.
SMDI (Simplified Message Desk Interface)
The Simplified Message Desk Interface (SMDI) protocol is intended to allow communication of basic message information between telephone systems and voicemail systems.
Asterisk supports SMDI, but given that this is an old protocol that runs across a serial connection, there are likely to be integration challenges. Support in various PBXs and other devices may be spotty. Still, it’s a fairly simple protocol, so it’s certainly worth testing out if you are considering using Asterisk as a voicemail replacement on an old PBX.
The following is not a detailed explanation of how to configure SMDI for Asterisk, but rather an introduction to the concepts, with some basic examples. If you are planning on implementing SMDI, you will need to write some complex dialplan logic and have a good understanding of how to interconnect systems via serial connections.
SMDI is enabled in Asterisk by the use of two options in the [general] section of the voicemail.conf file:
smdienable=yes
smdiport=/dev/ttyS0 ; or whatever serial port you are connecting your
; SMDI service to
Additionally, you will need an smdi.conf file in your /etc/asterisk folder to define the details of your SMDI configuration. It should look something like this (see the smdi.conf.sample file for more information on the available options):
[interfaces]
charsize=7
paritybit=even
baudrate=1200 ; hopefully a higher bitrate is supported
smdiport=/dev/ttyS0 ; or whatever serial port you’ll be using to handle
; SMDI messages on asterisk
[mailboxes] ; map incoming digit strings (typically DID numbers)
; to a valid mailbox@context in voicemail.conf
smdiport=/dev/ttyS0 ; first declare which SMDI port the following mailboxes
; will use
4169671111=1234@default
4165551212=9999@default
In the dialplan there are two functions that will be wanted in an SMDI configuration. The SMDI_MSG_RETRIEVE() function pulls the relevant message from the SMDI message queue. You need to pass the function a search key (typically the DID that is referred to in the message), and it will pass back an ID number that can be referenced by the SMDI_MSG() function:
SMDI_MSG_RETRIEVE(<smdi port>,<search key>[,timeout[,options]])
Once you have the SMDI message ID, you can use the SMDI_MSG() function to access various details about the message, such as the station , callerID , and type (the SMDI message type):
SMDI_MSG(<message_id>,<component>)
In your dialplan, you will need to handle the lookup of the SMDI messages that come in, to ensure that calls are handled correctly. For example, if an incoming call is intended for delivery to a mailbox, the message type might be one of B (for busy) or N (for un‐
answered calls). If, on the other hand, the call is intended to go to VoiceMailMain() because the caller wants to retrieve his messages, the SMDI message type would be D , and that would have to be handled.
