Le-tools.com - XL-Parser

Description

What is it

XL-Parser is a tool for data extraction and analysis. Input can be a directory, a file or multiple files, or the clipboard.

File formats: XL-Parser supports many file formats like plain text, unicode, text inside binaries, doc and docx, xls and xlsx, evt and evtx, pdf, msg and text inside zip files. You can even extract a file using a different parser. For example, you can extract data from a docx file, but open it as a zip, so you will have access to xml file inside it.
File filters: If the input is a directory, you can include the subfolder or not. You can also set a combinaison of filters: keyword or regex that match filename, size of the file, last modified time or last accessed time. Your combinaison of filters can be save for a later use.
Extraction: You can extract data using a combinaison of keyword or regex or use the special objects already set which are: IPv4, IPv6, URLs, Emails, Hostnames, Domain names, MAC addresses or credit card numbers. XL-Parser provides a lot of options for extraction and for report.
Web log analysis: One specialty of XL-Parser is web log analysis. When you analyse web logs, you can extract data using anything that is in the logs. For example, you can extract all requests related to a particular IP address, but you cannot search for requests related to an ISP, because this information is not in the log. XL-Parser provides you the way to do this and much more. First, XL-Parser will help you parse the log and build a SQlite database from it. When it's done, XL-Parser provides a bunch of functions to query the database and find suspicious activities. Cool!
Split logs: This goal of this function is to split logs but it can be used to split any text file, if you want to preserve the integrity of each line.

See documentation for more details.

How it works

XL-Parser can be installed using the installer or used as a standalone application. In order to start using it, just run the executable file from the start menu or the directory you install it.

If you installed it, you can also start the tool using the link in the Send To folder after right-clicking on any folder or file(s).

What do you need

Windows XP SP2 or newer

Screenshots

Main window

Extract - Special objects

Web log analysis

Split Logs

Window - Expression

Window - Expression tool

Window - Extract Expression options

Files - Extraction results

Window - Extraction report options

Window - Log format database

Window - Log format database - Add

Window - Log Analysis Filters

Window - Log Analysis Filters - Add

Window - Log Analysis Field Filters

Window - Log Analysis Progress

Database

Database - Query

Window - Saved Queries database

Window - Query database report options

Database - Suspicious Activities

Window - Suspicious Activities

Version History

Version	Comments
1.3.6 2020-01-12	New: Removed GeoIP database update.
1.3.5 2019-06-09	Fixed: Check update for GeoIP database was not working properly.
1.3.4	New: [Datetime database] New pattern %P (am/pm lowercase) in Database when creating a new Datetime object. Fixed: [Datetime database] Regex was not including upper cases when creating a new Datetime object with %p pattern (AM/PM uppercase). [Log format database] The May contain space option was not checked (when required) when editing a Log format. [Log format database] Too much spaces were sometimes added in the pattern when creating a new Log format.
1.3.3	Fixed: There were issues with the download of the GeoIP and the Resolve TLD databases when running autoconfiguration. There was an issue with the download of the Resolve TLD database from Settings window. Datetime format was not validated correctly when adding or editing a Log format.
1.3.2	Fixed: The version of the tool was not matching the current version. Unblessed reference error when downloading GeoIP2 database.
1.3.1	Fixed: GeoIP2 was generating fatal error when trying to resolve some IP addresses like non public addresses.
1.3	New: GeoIP is replaced by GeoIP2 (support for IPv6). GeoIP language can be set in Settings window. In Query database, testing the SQL query before execution is not required anymore. Fixed: When creating regex in create/edit Log format, IPv6 addresses were not correctly parsed and line was rejected if size was null (-). In Suspect activities detection: Number of long requests wasn't counted correctly in certain circumstances. "illegal backslash escape sequence in string" error.
1.2.2	Fixed: There was a problem in the install program. User directory was not created.
1.2.1	New: New: The XL-Parser version is now inserted in metadata for XLSX and HTML reports. Fixed: Starting XL-Parser from Windows Explorer (Send To...), with multiple files selected, was not working. When a database was not set (XL-Whois DB, GeoIP DB, MACOUI DB, Datetime DB, etc.), options were still available and error messages was displayed when process was started. Controls and options are now disabled if databases are not set. When adding an extension in File Formats, there was an error message (Not a scalar reference..., line 1029). Fixed: There was an issue with daylight saving time changes for last-accessed and last-modified time. Erroneous values were read from system. This problem occured with NTFS volume and if the computer was set to "Automatically adjust clock for daylight saving changes" (which is the default setting).
1.2	New: In Add/Edit Log Format window, the Fields may contain spaces option is now global and has effect on the regex (double-quotes are allowed in fields for non-Apache style log format). In Suspicious Activities detection, the SQL Query indicator is now tested on useragent field. When you want to change report options in Suspicious Activities, you must now click on the save button in the report window. In Suspicious Activities, you can now save and load different set of options. Fixed: In Search Database, there was a display bug with the auto validation of SQL query. SQL query must now be tested manually before executing the process. When selecting a destination database for creating log database, .db extension is added if not already there. Timezone name regex in Datetime conversion is now set to letters only. When guessing datetime format, there was an issue when format found was set as output only. This issue has been fixed and there is now a warning when there is more than one match. In File Order window (Web Log Analysis), datetime are now set to local timezone. In Search Database tab, GROUP BY button didn't give the right selection. In Suspicious Activities result window, there was in issue when double-clicking on Request length (nbr) indicator (Activities only).
1.1.1	Fixed: Issue with the use of operators in Extraction function (for expressions). Issues related to the use of non-ascii characters in filename. Display bug in the Query Database tab.
1.1	New: Reset function added to File Formats. Reset function added to Suspicious activities. In Extraction Results, a new function to get results only, without duplicates. The Request length indicator in Suspicious activities was splitted in two: Request length (nbr), Request length (max). Fixed: When editing a single expression, the operator was changes for OR. Expression window, when saving an expression, expression was saved but error occured. After a database and its files were moved, saved results file was crashing.
1.0	First release.

Translation

To translate XL-Parser to other languages:

In Settings window, use the Export Lang.ini function. The file will be saved in the same directory of the tool;
Open the file in any text editor like Notepad;
Translate each expression at the right of the = symbol;
- The expression on the left side is used by the tool to identify the expression so do not change it. Also, be sure to have a space between the = symbol and your expression (ex. Key = Value);
- Authorized characters are alphanumerics, spaces and these symbols: ",", ".", "-", "!", ",", "(" and ")". Any other character will be deleted;
- For some controls, string length must be the same as original. A longer string could be truncated if it doesn't match the length of the field;
- The value associated with the translatorName will be used in the About window to identify you as translator (if you want). You can also add your email or a short url (ex. YourName (youraddress@email.com));
Restart the tool so the strings in Lang.ini will be used instead of the default language.

If you translate the tool and you want to share, contact the author.

Available translations:

To install, save the appropriate Lang.ini file in the default folder of the tool (if you used the installer, it should be AppData. Otherwise, it's the same folder of the program).

No translation available for the moment.

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

Credits

For database:

This product includes GeoLite2 data created by MaxMind, available from https://www.maxmind.com.
OUI Database comes from the IEEE Standards Association. Link for the database is http://standards-oui.ieee.org/oui.txt.
The Public Suffix List come from https://publicsuffix.org. Link to the database is https://publicsuffix.org/list/public_suffix_list.dat.

For icons:

The main logo of XL-Parser has been created by Oxygen Team on term of the GNU Lesser General Public License.
Big filter list icon has been created by Double-J Design and is distributed on term of the CC Attribution-No Derivative 4.0 license.
Big config icon used in option windows has been created by IconLeak and is distributed as Linkware.
A lot of icons used for buttons come from the Blue Bits collection of Icojam (licensed as "Public Domain") like config, about, open file, delete, check update, import database, etc.
A lot of icons used for buttons come from the Farm Fresh Icons collection by Fatcow Web Hosting and are distributed on term of the CC Attribution 4.0 license.
Open folder icon, clipboard icon, open regex tool icon, has been created by PixelMixer and is distributed as freeware.
Filter icons (add, edit, delete) come from the Fugue Icons collection by Yusuke Kamiyamane (http://p.yusukekamiyamane.com) and are licensed under a Creative Commons Attribution 3.0 License.
Small up and down arrows has been created by Oxygen Team on term of the GNU Lesser General Public License.
Down arrow in Log format window have been created by VirtualLNK and is distributed as Linkware.
The "Open folder in Explorer" icon comes from Danish Royalty Free Icons by Jonas Rask Designand is distributed as Freeware.
The icon used for Process button has been created by Saki (Alexandre Moore) and is distributed on term of the GNU General Public License.
Stop icon has been created by emey87 (Manuel Lopez) and is distributed on term of the CC Attribution-No Derivative 4.0 license.
Documentation icon has been created by Oxygen Team on term of the GNU Lesser General Public License.
The check icon used in Configuration Wizard has been created by Cheezen (Anders Bjarnle) and is distributed as freeware.
The icon (128x128) used in Save query window comes from Yellowicon, has been created by Everaldo and is distributed on term of the GNU Lesser General Public License.

Download

Source code is hosted on SourceForge and GitHub. Binaries are hosted on SourceForge.

Filename	MD5	SHA256
XL-Parser 1.3.6 Setup.exe	c4161799dd9af9e5534cb71c5ddc0a05	ae60aec0df1206838ce38b379e553336ecede43c093e038415a84ebaa9310bfc
XL-Parser 1.3.6.zip	bd0fe76c76d8e7e5aed93181ea00f16e	0af948159fb7aba75b5791ae5242ea8a4c1c4b43e77d8f12060705db4377e442
XL-Parser 1.3.5 Setup.exe	9123e26b67bc0431532ad1f9189ffe23	bb684370229ba044de48049b22f6ffac0ffdfa1e73972ce7882bad8e8a518536
XL-Parser 1.3.5.zip	40035bc2131958dfdc6b1035f32d6c56	9b6d595792361d15dc38d70e022c3d53010754c75fb92c2058aec759a1809398

If you want to download the standalone version or an older version, check on SourceForge.

Additional content
Log format Database	[Updated 2019-03-02] This database is used for Web Log Analysis.
Datetime Database	[Updated 2019-03-02] This database is used in Log format database function and with the Split Logs function.
Custom Function - Resolve TLD	This function is used with Special objects extraction.

If you install many tools of the XL-Toolkit, you should follow these instructions.