Least number bits

Least Number of Bits (LNB) is a database storage technology, which was first published in February 2001, as a type of database management system internals abstract. It consists of a method and system for determining the absolute least number of bits required to represent and process information within any database input and output process.

The method includes an automated process for determining the most compressed form to represent each data item, including a file organization and access method that further facilitates transmission of only the few pertinent bits of data necessary to support any input output process.It pertains to databases, and more particularly, to the representation and storage of detailed data used in control systems and information systems software, including transaction and decision support processing involving highly compact data technology and very large databases.

Examples of LNB methods exist among various products known for fast indexing, fast searching, or any representation of data requiring hyper-compression for optimization. Although the complete set of methods is still proprietary, the technological foundation for these proprietary methods is openly available to mathematicians and technologists.

History

The notion of least number of bits technology was first proposed for artificial intelligence applications in the 1990’s as a means for encoding vast amounts of knowledge into the least amount of storage possible. The concepts were prototyped in the late 90’s to demonstrate significant value in various types of computer applications, particularly those faced with having to store and process extremely large volumes of data for analysis. The deployment of least number of bit methods facilitated efficient and effective use of available network bandwidths for global applications, and rendered conventional data warehousing technologies obsolete within the applications wherein LNB technology was introduced.

Architecture

The basic architecture of LNB technology places the compressed value of each data occurrence of a data field into a contiguous one dimensional array that disregards the word and byte boundaries of conventional storage technology. A one-dimensional array is a string of one or more compressed occurrences of a specific data field, or even a bit position of a specific data field in its compressed form. In contrast to techniques used in conventional file organizations, gaps are not permitted within a one-dimensional array. For example, a two character code such as state codes would require a maximum of six bits to represent each state. As such, the second six bit occurrence begins in the seventh and eighth bits of the same byte that houses the first occurrence. Due to the flexibility and portability of this architecture, the methods are easily transferable to application specific integrated circuits (ASIC), programmable chips (FPGA) or RAID hardware technology. The data dictionary itself establishes and maintains the key for all encryption and decryption of each data field value that exists within each one dimensional array, with full precision for the value of each data field.The architecture fully extends to support relational theory, including the data virtualization of tables, foreign keys, indices, and all types of joins, and in addition to non-key values, it provides for the compression of all key values as well as support for mathematical operations that can be certified against the data in its compressed form.

This technology was first unveiled within the private sector at DAMA in New York City on May 12 2000 at the Data Warehouse conference.