About me

Paolo Iannelli Picture

Paolo Iannelli

Sr. Software Engineer / Software Architect at LeaseWeb

Location
Amsterdam Area, Netherlands
Industry
Information Technology and Services
Interests
Python, Big Data, Scalability, High Availability, Performance
Description
Senior Software Engineer and Architect with more than 8 years experience.
Strong in critical thinking, problem solving and high performance architectures.
Paolo Iannelli Labs Rss

FIFO queues in Perl

Posted on : 27-02-2011 | By : Paolo Iannelli | In : Perl, Software Development

Tags: , , , ,

1

Most of the times, when there is a need to exchange data between two unrelated processes, comes handy to use FIFO queues (also referred as named pipes) to solve the problem.

But, what is a FIFO queue?

FIFO is an acronym for “First-in, First-out” and represent a concept that describes how data is organized and served. More specifically, when we talk about FIFO queues, we imagine a place where we send all our data, knowing that any successive read will access the oldest inserted data first, in a oldest-first fashion.

Why would we need to use a FIFO queue ?

Imagine you have to send some data to an exclusive device (for instance a COM/serial port) : you can’t have multiple processes busy at the same time with the port and, also, you have to handle any possible conflict other processes may cause (for instance, while your software is releasing for a small amount of time the resource, if another process take control of the resource, then you have to wait for an undefined amount of time for the resource to become available again). So, why not create a queue of commands to send via COM port and having only ONE process to do this job ?

Here FIFO queues are the best !

How this entire thing will work then ?

Let’s imagine the following setup : we have a server connected to a special card that receive data via COM port and print everything in a huge display.
We have then 5 applications that send requests, most likely their “application log”. We prepared for them a simple Perl script that is called whenever something critical happens.
We also prepared a script in Perl that do the real work, that is reading the FIFO queue and transmitting the data via COM port.

The application’s script will look like this :

As you can see, what it does is pretty simple : if the FIFO file (yes, a FIFO queue in linux, after all, is just a special file!) doesn’t exists, then it create it. Once is create it just take the entire arguments list and write it to the FIFO file. That’s it ! We force the write to file to avoid any buffering and to have data stored in real-time.

The server script will look like the following and is nothing different than the previous one. I added also a LOGFILE that stores all the data fetched from the FIFO list for debug purposes (but also to keep all logs in a aggregated file).

Also in this case, we make sure the FIFO file exists and then we initialize the COM port via the stty command. Once this is done we are ready to open the appropriate files (FIFO, Serial Port and Log File), read from the FIFO line by line and output the content respectively in the COM port or the log file : can’t be any simple!

That’s it ?

Well, I could say yes, but there are some things to mention.
If you try to launch the application’s logger script (the one that sends logs entries to the FIFO) without launching the other script, you will notice that your script freeze. That’s normal for FIFO queues : in fact, the write operation will wait until there is a reader attached to the FIFO queue. If there is no one reading from the FIFO, then Perl will pause the script execution until the reader is attached to it. So, if your reader deamon is dead, yes, your script will stuck !
How do you fix that ?
You can create a pid file upon launch of the script that read the FIFO queue and check this pid in your log script : if the pid is not running for some reasons, then you execute it just before writing to the FIFO.

The only case where this solutions won’t work, is when there is a serious problem with the reader script (for instance it can’t open the COM port, no disk space for logging or the script is simply broken).

On some Unix systems, it may be necessary to use mkfifo command instead of mknod. You may simply change the system call to :

if you experience problems in the FIFO creation.

Now, that’s it!

I hope this was informative enough for you and if you have any questions just leave a comment below !