By Lamie Saif.
The amount of data generated by the new models of CCTV cameras is huge and so its storage requires some planning. So, how can this best be done? Before we choose the best hardware and software for optimum CCTV data storage, we need to take into account the following parameters:
- Camera resolution.
- Number of frames per second.
- Level of compression.
- Number of cameras.
- Storage period.
- Level of data safety
The first four parameters define the size of storage needed per day. The more storage needed per day, the higher these parameters will be, except for their compression level. The lower the compression level used, the more storage space will be needed, bearing in mind that higher compression levels can reduce the quality of the retrieved CCTV images.
The other two parameters are related to the storage period, storage policy and data manipulation. The storage period is determined by the CCTV monitoring regulations in the country where the system is being used, but let us take the example of 30 days, and calculate the storage needed for a two-megapixel camera per day with the following configuration :
- Video stream MPEG4
- Video quality HIGHEST
- FPS 25
With 24 recording hours per day, the storage needed for this camera is about 200 GB /day. Now for 10 cameras of the same type, the storage needed per day is about 2 TB, which means sixty terabytes of total storage for thirty days. This means that we need a 20 X 3 TB hard drive, and we need to have a safety margin; so the capacity should be increased to a 24 X 3 TB hard drive, which is a good margin.
How should these hard disks be arranged, and what type of hard disks should we use? To answer this question, we need to know what types of hard disks are presently available. There are four main types of hard disks: ATA, SATA (serial ATA), SCSI and SAS (serial SCSI). The ATA and SCSI are the older types of hard disks, while SATA and SAS are newer.
ATA and SATA are low-priced hard disks so they are good for non-critical storage. SCSI and SAS are the high–end, hard disks so they are good for mission-critical storage because of their high level of performance. Each of these hard disks (ATA, SATA, SCSI, SAS) is connected to their host through what is called an HBA (Host Bus Adapter) and a HBA has the same relative names as the hard disks.
ATA HBA can address 2 hard disks with a maximum transmission speed rateof 1064 Mb/sec for the latest version; and SATA HBA can address 4 hard disks with a maximum transmission speed rate of 6000 Mb/sec for its latest version. SCSI HBA can address 15 hard disks with a maximum transmission speed rate of 5120 Mb/sec for its latest version. SAS HBA can address 65000 hard disks with a maximum transmission speed rate of 6000 Mb/sec for its latest version.
Now, back to our example and what to choose for it. To start with, we must drop out of the ATA solution, because of the very low number of addresses, so the other possible solutions are:
Three servers, each with two SATA HBA to address 8 hard disks per server and the required number of total hard disks is reached. But this solution needs more hardware, and there is no possibility of future expansion unless we add a new server.
One server with two SCSI HBA’s can address 12 hard disks, and the required number of total hard disks is reached with the possibility of adding six more hard disks for future expansion.
One server that has one SAS HBA that can address all of the 24 hard disks and the required number of total hard disks reached, with the possibility of adding what is needed for future expansion.
These Hard Disks Can Be Arranged In Two Main Ways
The first arrangement is very simple. Called JBOD (Just a Bunch Of Drives), there is no link between disks, so this will make our data-safety level very low. But, the good part is, in case of a hard disk crash, we only lose the data in that crashed hard disk. To improve the safety level of a JBOD arrangement, use a secondary type of storage, such as a tape/disk backup or data replication connected to a replication server.
The second arrangement is the RAID, which is more complex and more flexible with higher data safety in all, except for one type of RAID that provides less data safety than the JBOD arrangement.
There are primary types of RAID arrangements, as well as ones that are more advanced. The four primary types of RAID that are supported in the market today, include:
- RAID 0, which is a set of similar hard disks that work like a single, large hard disk with write process striped evenly on every one. This means that the write/read process is very fast in this type of RAID, but there is the risk of losing all of the data if just one hard disk crashes, so this type of RAID is less reliable than the JOBD arrangement.
- RAID 1, which is a two, hard disk arrangement with the same data written on both hard disks at the same time, so it is a mirrored, hard disk arrangement. This type of RAID is very reliable, but the cost is high because the size of the storage is only equal to one hard disk, and there is no gain in write speed. However, the read speed is faster.
- RAID 5, which is a set of similar hard disks that work like a single large hard disk with the write process striped evenly on all of them. However, the data is protected with a parity check that is distributed across all of the disks, which means more safety. But the storage size will be N1 (N being the number of hard disks in RAID 5). The write process is slow because, to get the parity check for RAID 5, there should be write, then read, to establish the parity check before the rewrite of the data. With parity check, the RAID 5 can tolerate losing one hard disk with no data loss.
- RAID 6 is a set of similar hard disks that work like a single, large hard disk with the write process striped evenly on all of them. The data is protected with two parity checks that are distributed on all of these hard disks, providing higher safety than RAID 5, but the storage size will be N2 (N being the number of hard disks in RAID 6). The write process is slow because, to get the parity check for RAID 6, there should be write then read, to establish the two parity checks, before the rewrite of the data. By this means, the RAID 6 can tolerate losing two hard disks with no data loss.
Advanced RAID arrangement
The advanced Raid arrangement is the combination of two types of the primary RAID, so as to remove or minimize the drawbacks that we have highlighted in the primary RAID. For example:
RAID 01 consists of two, equal sets of hard disks with each set configured as RAID 0, then reconfigured as RAID 1.
RAID 10 consists of two equal sets of hard disks with the first hard disk from set 1, and the first hard disk from set 2 configured as RAID 1 and so on, until the last hard disk from set 1 and the last hard disk from set 2 are configured as RAID 1. Then, the two sets are configured as RAID 0 with the speed of read and write of RAID 0, and the reliability of RAID 1. The storage size will be N/2 (N being the number of all the hard disks in this RAID arrangement). Mathematical calculations show that RAID 10 is more reliable than RAID 01.
This kind of arrangement can be repeated as a means of increasing storage capacity from say, RAID 05 to RAID 50, or from RAID 15 to RAID 51.
A concluding article about storage systems will follow in a later issue.
Lamie Saif
S. Data & Monitoring Consulting
www.lssurveillance.com
lasaif@lssurveillance.com
For more information, visit: http://www.lssurveillance.com/CCTVCalculator.html
