Show simple item record

dc.contributor.authorJain, Shubham
dc.contributor.authorde Buitléir, Amy
dc.contributor.authorFallon, Enda
dc.date.accessioned2021-10-19T10:56:56Z
dc.date.available2021-10-19T10:56:56Z
dc.date.copyright2020
dc.date.issued2020-11-30
dc.identifier.citationJain, S., de Buitléir, A., Fallon, E. (2020) Unsupervised Noise Detection in Unstructured data for Automatic Parsing. 16th International Conference on Network and Service Management (CNSM), 2020, pp. 1-5, doi: 10.23919/CNSM50824.2020.9269096.en_US
dc.identifier.urihttp://research.thea.ie/handle/20.500.12065/3722
dc.description.abstractThe telecommunications industry makes extensive use of data extracted from logs, alarms, traces, diagnostics, and other monitoring devices. Analyzing the generated data requires that the data be parsed, re-structured, and re-formatted. Developing custom parsers for each input format is labor-intensive and requires domain knowledge. In this paper, we describe a novel unsupervised text processing pipeline to automatically detect and label relevant data and eliminate noise using Levenshtein similarity and Agglomerative clustering. We experiment with different similarity and clustering algorithms on a selection of common data formats to verify the accuracy of the proposed technique. The results suggest that the proposed methodology has higher accuracy.en_US
dc.formatPDFen_US
dc.language.isoengen_US
dc.publisherIEEEen_US
dc.relation.ispartof2020 16th International Conference on Network and Service Management (CNSM)en_US
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.subjectUnsupervised data miningen_US
dc.subjectInformation extractionen_US
dc.subjectClusteringen_US
dc.subjectSimilarityen_US
dc.titleUnsupervised noise detection in unstructured data for automatic parsingen_US
dc.conference.date2020-11-02
dc.conference.hostIEEEen_US
dc.conference.locationIzmir, Turkeyen_US
dc.contributor.affiliationAthlone Institute of Technologyen_US
dc.contributor.sponsorIrish Research Council Enterprise Partnership Scheme Postgraduate Scholarship 2020en_US
dc.description.peerreviewyesen_US
dc.identifier.doi10.23919/CNSM50824.2020.9269096.en_US
dc.identifier.orcidhttps://orcid.org/ 0000-0002-0913-3948en_US
dc.identifier.orcidhttps://orcid.org/ 0000-0001-8359-0920en_US
dc.identifier.orcidhttps://orcid.org/ 0000-0002-8300-5813en_US
dc.rights.accessrightsinfo:eu-repo/semantics/openAccessen_US
dc.subject.departmentSoftware Research Institute AITen_US
dc.type.versioninfo:eu-repo/semantics/acceptedVersionen_US
dc.relation.projectidProject EPSPG/2020/7en_US


Files in this item

Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record

Attribution-NonCommercial-NoDerivatives 4.0 International
Except where otherwise noted, this item's license is described as Attribution-NonCommercial-NoDerivatives 4.0 International