{"id":30125,"date":"2025-10-04T05:52:34","date_gmt":"2025-10-04T00:52:34","guid":{"rendered":"https:\/\/kmwllc.com\/?p=30125"},"modified":"2025-11-06T20:20:27","modified_gmt":"2025-11-06T15:20:27","slug":"whats-the-best-way-to-do-entity-extraction-for-search","status":"publish","type":"post","link":"https:\/\/kmwllc.com\/index.php\/2025\/10\/04\/whats-the-best-way-to-do-entity-extraction-for-search\/","title":{"rendered":"What&#8217;s the best way to do entity extraction for search?"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"30125\" class=\"elementor elementor-30125\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-5260c7c elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5260c7c\" data-element_type=\"section\" data-e-type=\"section\">\r\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-thegem\"><div class=\"elementor-row\">\r\n\t\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-f697649\" data-id=\"f697649\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2598a50 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-post-info\" data-id=\"2598a50\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"post-info.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<ul class=\"elementor-inline-items elementor-icon-list-items elementor-post-info\">\n\t\t\t\t\t\t\t\t<li class=\"elementor-icon-list-item elementor-repeater-item-91a0f52 elementor-inline-item\" itemprop=\"datePublished\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"elementor-icon-list-text elementor-post-info__item elementor-post-info__item--type-date\">\n\t\t\t\t\t\t\t\t\t\t<time>October 4, 2025<\/time>\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t<\/li>\n\t\t\t\t<\/ul>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4829f73 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"4829f73\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"styled-subtitle elementor-heading-title elementor-size-default\">Comparing the Effectiveness of Entity Extraction between NLP and LLMs<\/div>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3906aea elementor-author-box--image-valign-top flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-author-box\" data-id=\"3906aea\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"author-box.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-author-box\">\n\t\t\t\t\t\t\t<div  class=\"elementor-author-box__avatar\">\n\t\t\t\t\t<img src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/Jacob.jpg\" alt=\"Picture of Jacob Squatrito\" loading=\"lazy\">\n\t\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"elementor-author-box__text\">\n\t\t\t\t\t\t\t\t\t<div >\n\t\t\t\t\t\t<div class=\"elementor-author-box__name\">\n\t\t\t\t\t\t\tJacob Squatrito\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-author-box__bio\">\n\t\t\t\t\t\t<p>Search & AI Engineer at KMW Technology<\/p>\n\t\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7a1c601 elementor-widget-divider--view-line flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-divider\" data-id=\"7a1c601\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"divider.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-divider\">\n\t\t\t<span class=\"elementor-divider-separator\">\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-54998f4 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"54998f4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Entities are really important for search, but what\u2019s the best way to do it?<\/span><\/p><p><span style=\"font-weight: 400;\">The ability to analyze a piece of text and identify the key entities within it can have lots of practical search uses like improving search relevancy; enabling faceting and filtering classifying documents and even obfuscation of sensitive data.<\/span><\/p><p><span style=\"font-weight: 400;\">As a natural language problem, entity extraction is not new but it\u2019s always been tricky to do well. There are lots of traditional NLP models for entity extraction and recently LLMs have shown promising abilities too. For search applications we are typically trying to balance excellent entity extraction along with operational needs like running fast and not consuming too many resources. So what is the best way to do entity extraction for modern search applications?<\/span><\/p><p><span style=\"font-weight: 400;\">In this blog post we are going to compare traditional NLP models with LLMs to see how they measure up. We&#8217;ll focus on extracting the names of people, organizations, or locations within a body of text. We\u2019ll discuss ways entity extraction can improve the search experience, analyze the performance of traditional models and large language models, run a few experiments, and conclude with a review of our findings<\/span><\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8717084 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"8717084\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-large\">A Simple Example<\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5831465 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"5831465\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Imagine we have four documents we want to index:<\/span><\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2e5a0c6 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-code-highlight\" data-id=\"2e5a0c6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-json \">\n\t\t\t\t<code readonly=\"true\" class=\"language-json\">\n\t\t\t\t\t<xmp>{\r\n  \u201cid\u201d: 1\r\n  \u201ctext\u201d: \u201cWill, are you going to the store today?\u201d\r\n}\r\n\r\n{\r\n  \u201cid\u201d: 2\r\n  \u201ctext\u201d: \u201cWill you go to the store today?\u201d\r\n}\r\n\r\n{\r\n  \u201cid\u201d: 3\r\n  \u201ctext\u201d: \u201cI hope you will join us.\u201d\r\n}\r\n\r\n{\r\n  \u201cid\u201d: 4\r\n  \u201ctext\u201d: \u201cIs Hope going to be joining us?\u201d\r\n}\r\n\r\n\r\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a9743f6 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"a9743f6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Let\u2019s say you want to search for documents referencing a certain person. In some cases, you might get away with just searching for their name, as-is, in <\/span><span style=\"font-weight: 400;\">text<\/span><span style=\"font-weight: 400;\">. But, when you search for names like Will or Hope, your search engine will likely return documents that use these words in a different context. As you can see, \u201cwill\u201d is found in documents 1, 2, and 3, but only document 1 actually references someone named Will. We run into a similar issue when searching for \u201chope\u201d as well.\u00a0<\/span><\/p><p><span style=\"font-weight: 400;\">Having some irrelevant documents returned is, of course, not ideal. But in practice, the problem may be more than just a minor nuisance. Terms like Will and Hope could be used more often as English words rather than names. Documents that naturally use the word \u201cwill\u201d multiple times might score higher than documents mentioning a person named Will. Making matters even worse, the text might reference Will with pronouns instead of stating his name repeatedly, further decreasing the document\u2019s search score. <\/span><\/p><p><span style=\"font-weight: 400;\">So\u2026 just running a search on <\/span><span style=\"font-weight: 400;\">\u201ctext\u201d<\/span><span style=\"font-weight: 400;\"> won\u2019t always suffice. To avoid manually pruning through your search results to remove the irrelevant documents, you\u2019ll want to enrich your content before indexing.\u00a0<\/span><\/p><p><span style=\"font-weight: 400;\">With entity extraction, we can enrich our documents like so:<\/span><\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3a9e936 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-code-highlight\" data-id=\"3a9e936\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-json \">\n\t\t\t\t<code readonly=\"true\" class=\"language-json\">\n\t\t\t\t\t<xmp>{\r\n  \u201cid\u201d: 1\r\n  \u201ctext\u201d: \u201cWill, are you going to the store today?\u201d\r\n  \u201cpeople\u201d: [\u201cWill\u201d]\r\n}\r\n\r\n{\r\n  \u201cid\u201d: 2\r\n  \u201ctext\u201d: \u201cWill you go to the store today?\u201d\r\n}\r\n\r\n{\r\n  \u201cid\u201d: 3\r\n  \u201ctext\u201d: \u201cI hope you will join us.\u201d\r\n}\r\n\r\n{\r\n  \u201cid\u201d: 4\r\n  \u201ctext\u201d: \u201cIs Hope going to be joining us?\u201d\r\n  \u201cpeople\u201d: [\u201cHope\u201d]\r\n}\r\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c95a3aa flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"c95a3aa\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">If we extract entities from the text, we can attach lists of names mentioned to the document. Now, if we want to search for documents that reference somebody named Will, we would search on <\/span><span style=\"font-weight: 400;\">\u201cpeople\u201d<\/span><span style=\"font-weight: 400;\"> instead of <\/span><span style=\"font-weight: 400;\">\u201ctext\u201d<\/span><span style=\"font-weight: 400;\">. This allows us to ensure documents like 2 and 3 don\u2019t hinder our search process or contaminate our results. This is definitely a simple use case &#8211; different use cases will certainly have unique challenges to overcome.<\/span><\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4722b30 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"4722b30\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-large\">Our Entity Extraction Approach<\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9370169 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"9370169\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">There exist many statistical\/rules-based models for entity extraction and named entity recognition. For our testing, we used Apache OpenNLP\u2019s <\/span><a href=\"https:\/\/opennlp.sourceforge.net\/models-1.5\/\"><span style=\"font-weight: 400;\">pretrained models<\/span><\/a><span style=\"font-weight: 400;\"> and Stanford CoreNLP\u2019s built-in named entity recognition models. But they aren\u2019t perfect, and with the ever increasing popularity of large language models (LLMs), we thought it would be interesting to see how an LLM performs entity extraction compared against these legacy approaches. <\/span><\/p><p><span style=\"font-weight: 400;\">We theorized that an LLM had the potential to extract certain names that more traditional models may be likely to miss. (In particular, we figured an LLM would be more likely to extract a \u201cnewer\u201d name that was uncommon when the traditional models were trained.) But, we also believed an LLM could potentially get \u201cdistracted\u201d and underperform the traditional models on longer pieces of text.<\/span><\/p><p><span style=\"font-weight: 400;\">In order to test our theories, we ran four different experiments:<\/span><\/p><ol><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\"><strong>Baseline:<\/strong> We started by evaluating the performance of OpenNLP, CoreNLP, and Google\u2019s <\/span><a href=\"https:\/\/ollama.com\/library\/gemma3\"><span style=\"font-weight: 400;\">gemma3<\/span><\/a><span style=\"font-weight: 400;\">, a popular and capable open-source LLM. The default version of the model is about 3 GB in size and has roughly 4 billion parameters, making it suitable for use on modern hardware. For each model, we evaluated its precision, recall, and F1 score.<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\"><strong>Number of Parameters:<\/strong> We introduced two new variants of gemma3 with a different number of parameters. We discussed how using more \/ less parameters appeared to affect the results.<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\"><strong>Two Pass<\/strong>: Using the two strongest models &#8211; CoreNLP and gemma3 &#8211; we instructed gemma3 to observe and edit the output of CoreNLP as it saw fit.\u00a0<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\"><strong>Alternate Model:<\/strong> We evaluated another popular model, <\/span><a href=\"https:\/\/ollama.com\/library\/deepseek-r1\"><span style=\"font-weight: 400;\">deepseek-r1<\/span><\/a><span style=\"font-weight: 400;\">, against OpenNLP and CoreNLP, to see if there were any notable differences.<\/span><\/li><\/ol><p><span style=\"font-weight: 400;\">For all experiments, we used a publicly available and fully annotated <\/span><a href=\"https:\/\/github.com\/juand-r\/entity-recognition-datasets\/blob\/master\/data\/wikigold\/CONLL-format\/data\/wikigold.conll.txt\"><span style=\"font-weight: 400;\">wikigold dataset<\/span><\/a><span style=\"font-weight: 400;\">, allowing us to evaluate the models against a source of truth. Across the 140+ wiki articles, there were roughly 3,000 words annotated as a person, organization, or location.\u00a0<\/span><\/p><p><span style=\"font-weight: 400;\">As part of our evaluation, we worked with <\/span><a href=\"https:\/\/github.com\/kmwtechnology\/lucille\"><span style=\"font-weight: 400;\">Lucille<\/span><\/a><span style=\"font-weight: 400;\">, our open-source Search ETL solution that allowed us to pass text through entity extraction processes and update each document with the output. We created a custom Connector to process the wikigold dataset into Lucille, including the article\u2019s text as well as lists of the annotated (or \u201cgold\u201d) people, organization, and location names.\u00a0<\/span><\/p><p><span style=\"font-weight: 400;\">To perform entity extraction, we used three different Lucille Stages. We created a custom Stage to extract people, organization, and location names using OpenNLP. We did the same for CoreNLP as well. To work with an LLM, we used Lucille\u2019s PromptOllama Stage, which allows you to provide parts (or all) of a document to a compatible LLM for generic enrichment. The model was instructed to read the source text and output a JSON object including the names of people, organizations, and locations mentioned in the document. Lucille then integrated the model\u2019s JSON response into the document. The models did not have access to the output of other Stages &#8211; they only saw the source text.\u00a0<\/span><\/p><p><span style=\"font-weight: 400;\">Here\u2019s an example of what a finalized document looked like, after we normalized the output for evaluation:<\/span><\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bc53fdf flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-code-highlight\" data-id=\"bc53fdf\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript \">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>{\r\n  \u201ctext\u201d: \u201c010 is the tenth album from Japanese Punk Techno band The Mad Capsule Markets . This album proved to be more commercial and more techno-based than Osc-Dis , with heavily synthesized songs like Introduction 010 and Come . Founding member Kojima Minoru played guitar on Good Day , and Wardanceis cover of a song by UK post punk industrial band Killing Joke . XXX can of This had a different meaning , and most people did n't understand what the song was about . it was later explained that the song was about Cannabis ( ' can of this ' sounding like Cannabis when said faster ) it is uncertain if they were told to change the lyric like they did on P.O.P and HUMANITY . UK Edition came with the OSC-DIS video , and most of the tracks were re-engineered .\u201d\r\n  \u201copenNLP_people\u201d: []\r\n  \u201ccoreNLP_people\u201d: [\u201ckojima\u201d, \u201cminoru\u201d]\r\n  \u201collama_people\u201d: []\r\n  \u201cgold_people\u201d: [\u201ckojima\u201d, \u201cminoru\u201d]\r\n  \u201copenNLP_organizations\u201d: [\u201cuk\u201d, \u201ckilling\u201d, \u201cjoke\u201d, \u201cfounding\u201d]\r\n  \u201ccoreNLP_organizations\u201d: []\r\n  \u201collama_organizations\u201d: [\u201cthe\u201d, \u201cmad\u201d, \u201ccapsule\u201d, \u201cmarkets\u201d, \u201cmeta\u201d, \u201ckilling\u201d, \u201cjoke\u201d]\r\n  \u201cgold_organizations\u201d: [\u201cthe\u201d, \u201ckilling\u201d, \u201cmad\u201d, \u201cmarkets\u201d, \u201ccapsule\u201d, \u201cjoke\u201d]\r\n  \u201copenNLP_locations\u201d: []\r\n  \u201ccoreNLP_locations\u201d: [\u201cuk\u201d]\r\n  \u201collama_locations\u201d: [\u201cuk\u201d]\r\n  \u201cgold_locations\u201d: [\u201cuk\u201d]\r\n}\r\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b5e3211 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"b5e3211\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">For the purposes of evaluation, we did not index the documents into a search engine. Instead, they were indexed into a CSV, which stored the original text, annotated entities, and output from each model. We then ran a custom script to analyze the models\u2019 performance. Using the annotated people, organizations, and locations from the wikigold dataset, we were able to compute some key metrics for each model:<\/span><\/p><ul><li aria-level=\"1\"><b>Precision<\/b><span style=\"font-weight: 400;\"> &#8211; What percentage of the person\/organization\/location names output by a model were annotated as such in the dataset?<\/span><\/li><\/ul><ul><li aria-level=\"1\"><b>Recall<\/b><span style=\"font-weight: 400;\"> &#8211; What percentage of the annotated person\/organization\/location names in the dataset were output by the model?<\/span><\/li><\/ul><ul><li style=\"font-weight: 400;\" aria-level=\"1\"><b>F1 <\/b><span style=\"font-weight: 400;\">&#8211; The \u201charmonic mean\u201d of precision and recall. Considered a solid overall indicator of a model\u2019s performance.\u00a0<br \/><\/span><\/li><li><b>Unique Gold Words<\/b><span style=\"font-weight: 400;\"> &#8211; How many gold names did a model mention that no other model did?<\/span><\/li><\/ul>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b827d24 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"b827d24\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-large\">ExperIminets &amp; Results<\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-719512e flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"719512e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-medium\">Experiment 1: Number of Parameters<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8861a2a flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"8861a2a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">We began by creating a pipeline that used three different models: OpenNLP, CoreNLP, and gemma3. Each model ran independently of the other, meaning they were not aware of each other\u2019s output. Here are the precision, recall, and F1 scores for each model:<\/span><\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a33b483 elementor-arrows-position-inside elementor-pagination-position-outside flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image-carousel\" data-id=\"a33b483\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;slides_to_show&quot;:&quot;1&quot;,&quot;navigation&quot;:&quot;both&quot;,&quot;autoplay&quot;:&quot;yes&quot;,&quot;pause_on_hover&quot;:&quot;yes&quot;,&quot;pause_on_interaction&quot;:&quot;yes&quot;,&quot;autoplay_speed&quot;:5000,&quot;infinite&quot;:&quot;yes&quot;,&quot;effect&quot;:&quot;slide&quot;,&quot;speed&quot;:500}\" data-widget_type=\"image-carousel.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-image-carousel-wrapper swiper\" role=\"region\" aria-roledescription=\"carousel\" aria-label=\"Image Carousel\" dir=\"ltr\">\n\t\t\t<div class=\"elementor-image-carousel swiper-wrapper\" aria-live=\"off\">\n\t\t\t\t\t\t\t\t<div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"1 of 3\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp1gemma3.png\" alt=\"exp1gemma3\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"2 of 3\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp1opennlp.png\" alt=\"exp1opennlp\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"3 of 3\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp1corenlp.png\" alt=\"exp1corenlp\" \/><\/figure><\/div>\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-swiper-button elementor-swiper-button-prev\" role=\"button\" tabindex=\"0\">\n\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"eicon-chevron-left\"><\/i>\t\t\t\t\t<\/div>\n\t\t\t\t\t<div class=\"elementor-swiper-button elementor-swiper-button-next\" role=\"button\" tabindex=\"0\">\n\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"eicon-chevron-right\"><\/i>\t\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\t\t<div class=\"swiper-pagination\"><\/div>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0d0ebb4 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"0d0ebb4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Clearly, CoreNLP is the strongest contender here, with gemma3 in a close second. Both had similar F1 scores of about ~0.75. OpenNLP wasn\u2019t the strongest contender with a lower F1 score. We should also consider the latency associated with running the LLM.<\/span><\/p><p><span style=\"font-weight: 400;\">On average, gemma3 took about 8 seconds to respond per Document, slowing the pipeline down substantially. (The experiment was run on an Apple M1 Pro with 16 GB of RAM.)<\/span><\/p><p><span style=\"font-weight: 400;\">We also analyzed the effect of text length on model performance. Here, we considered just the top performing models &#8211; CoreNLP and gemma3. We calculated the same metrics (precision, recall, F1) <\/span><i><span style=\"font-weight: 400;\">for each<\/span><\/i><span style=\"font-weight: 400;\"> wiki article. There were a few articles with 900+ words that we excluded to avoid skewing the results. We also excluded articles with less than 100 words. Since these documents were very short, they usually just didn\u2019t reference names of a certain type. As a result, the model scores were primarily either zero or one, which made the results very volatile and difficult to observe:<\/span><\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-461ca00 elementor-arrows-position-inside elementor-pagination-position-outside flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image-carousel\" data-id=\"461ca00\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;slides_to_show&quot;:&quot;1&quot;,&quot;navigation&quot;:&quot;both&quot;,&quot;autoplay&quot;:&quot;yes&quot;,&quot;pause_on_hover&quot;:&quot;yes&quot;,&quot;pause_on_interaction&quot;:&quot;yes&quot;,&quot;autoplay_speed&quot;:5000,&quot;infinite&quot;:&quot;yes&quot;,&quot;effect&quot;:&quot;slide&quot;,&quot;speed&quot;:500}\" data-widget_type=\"image-carousel.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-image-carousel-wrapper swiper\" role=\"region\" aria-roledescription=\"carousel\" aria-label=\"Image Carousel\" dir=\"ltr\">\n\t\t\t<div class=\"elementor-image-carousel swiper-wrapper\" aria-live=\"off\">\n\t\t\t\t\t\t\t\t<div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"1 of 6\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp1textlength_gemma3_f1.png\" alt=\"exp1textlength_gemma3_f1\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"2 of 6\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp1textlength_corenlp_f1.png\" alt=\"exp1textlength_corenlp_f1\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"3 of 6\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp1textlength_gemma3_recall.png\" alt=\"exp1textlength_gemma3_recall\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"4 of 6\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp1textlength_corenlp_recall.png\" alt=\"exp1textlength_corenlp_recall\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"5 of 6\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp1textlength_gemma3_precision.png\" alt=\"exp1textlength_gemma3_precision\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"6 of 6\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp1textlength_corenlp_precision.png\" alt=\"exp1textlength_corenlp_precision\" \/><\/figure><\/div>\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-swiper-button elementor-swiper-button-prev\" role=\"button\" tabindex=\"0\">\n\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"eicon-chevron-left\"><\/i>\t\t\t\t\t<\/div>\n\t\t\t\t\t<div class=\"elementor-swiper-button elementor-swiper-button-next\" role=\"button\" tabindex=\"0\">\n\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"eicon-chevron-right\"><\/i>\t\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\t\t<div class=\"swiper-pagination\"><\/div>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4d8ec49 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"4d8ec49\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">As expected, the data here was a bit scattered. Since many points overlapped at the top and bottom of each chart, we did include lines of best fit to help visualize the overall trend. However, these lines should be interpreted cautiously, as they all had a <\/span><i><span style=\"font-weight: 400;\">very<\/span><\/i><span style=\"font-weight: 400;\"> low R-squared value. In other words, the length of a piece of text shouldn\u2019t be used to singularly predict the recall, precision, or F1 score you\u2019ll get from a model.<\/span><\/p><p><span style=\"font-weight: 400;\">Again, these charts should be observed with caution, as there were a multitude of factors at play here. But, it does seem reasonable to suggest that there was some sort of relationship between longer text and LLM underperformance. CoreNLP, on the other hand, appears to have been remarkably consistent.<\/span><\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b244a1e flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"b244a1e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-medium\">Experiment 2: Number of Parameters<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3ee8fc6 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"3ee8fc6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Our next pipeline included five different models (OpenNLP, CoreNLP, and three variants of gemma3). In addition to the previous pipeline\u2019s models, we added a smaller variant of gemma3, <\/span><a href=\"https:\/\/ollama.com\/library\/gemma3:1b\"><span style=\"font-weight: 400;\">gemma3:1b<\/span><\/a><span style=\"font-weight: 400;\">, and a larger variant of gemma3, <\/span><a href=\"https:\/\/ollama.com\/library\/gemma3:12b\"><span style=\"font-weight: 400;\">gemma3:12b<\/span><\/a><span style=\"font-weight: 400;\">. As we mentioned above, <\/span><a href=\"https:\/\/ollama.com\/library\/gemma3:latest\"><span style=\"font-weight: 400;\">gemma3<\/span><\/a><span style=\"font-weight: 400;\">, had 4 billion parameters and was a little more than 3 GB in size. The smaller variant, gemma3:1b, had 1 billion parameters and was less than 1 GB in size. The larger variant, <\/span><a href=\"https:\/\/ollama.com\/library\/gemma3:12b\"><span style=\"font-weight: 400;\">gemma3:12b<\/span><\/a><span style=\"font-weight: 400;\">, had 12 billion parameters and was about 8 GB in size.<\/span><\/p><p><span style=\"font-weight: 400;\"> Here are the results we found:<\/span><\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-23914b5 elementor-arrows-position-inside elementor-pagination-position-outside flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image-carousel\" data-id=\"23914b5\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;slides_to_show&quot;:&quot;1&quot;,&quot;navigation&quot;:&quot;both&quot;,&quot;autoplay&quot;:&quot;yes&quot;,&quot;pause_on_hover&quot;:&quot;yes&quot;,&quot;pause_on_interaction&quot;:&quot;yes&quot;,&quot;autoplay_speed&quot;:5000,&quot;infinite&quot;:&quot;yes&quot;,&quot;effect&quot;:&quot;slide&quot;,&quot;speed&quot;:500}\" data-widget_type=\"image-carousel.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-image-carousel-wrapper swiper\" role=\"region\" aria-roledescription=\"carousel\" aria-label=\"Image Carousel\" dir=\"ltr\">\n\t\t\t<div class=\"elementor-image-carousel swiper-wrapper\" aria-live=\"off\">\n\t\t\t\t\t\t\t\t<div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"1 of 5\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp2_gemma3_12b-1.png\" alt=\"exp2_gemma3_12b\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"2 of 5\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp2_gemma3-1.png\" alt=\"exp2_gemma3\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"3 of 5\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp2_gemma3_1b-1.png\" alt=\"exp2_gemma3_1b\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"4 of 5\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp2_opennlp.png\" alt=\"exp2_opennlp\" \/><\/figure><\/div><div class=\"swiper-slide\" role=\"group\" aria-roledescription=\"slide\" aria-label=\"5 of 5\"><figure class=\"swiper-slide-inner\"><img class=\"swiper-slide-image\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp2_corenlp.png\" alt=\"exp2_corenlp\" \/><\/figure><\/div>\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-swiper-button elementor-swiper-button-prev\" role=\"button\" tabindex=\"0\">\n\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"eicon-chevron-left\"><\/i>\t\t\t\t\t<\/div>\n\t\t\t\t\t<div class=\"elementor-swiper-button elementor-swiper-button-next\" role=\"button\" tabindex=\"0\">\n\t\t\t\t\t\t<i aria-hidden=\"true\" class=\"eicon-chevron-right\"><\/i>\t\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\t\t\t\t\t<div class=\"swiper-pagination\"><\/div>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-46b8c90 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"46b8c90\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p>Again, gemma3 and CoreNLP are the strongest contenders here. The smallest LLM, gemma3:1b, didn\u2019t do well &#8211; not only did it have a poor F1 score, but we found it was actually struggling to follow our instructions. Surprisingly, the largest LLM, gemma3:12b, was actually a bit worse than the medium variant, gemma3. Compared to gemma3, gemma3:12b had a somewhat higher precision but a notably lower recall. It seems that this larger model was a bit too cautious when engaging with the source text.<\/p><p id=\"ember375\" class=\"ember-view reader-text-block__paragraph\">For this experiment, we also calculated the number of \u201cgold\u201d words that were uniquely mentioned by each model. <\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-01e24bd flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\" data-id=\"01e24bd\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" width=\"793\" height=\"427\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp2_goldwords.png\" class=\"attachment-large size-large wp-image-30235\" alt=\"\" srcset=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp2_goldwords.png 793w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp2_goldwords-300x162.png 300w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp2_goldwords-768x414.png 768w\" sizes=\"(max-width: 793px) 100vw, 793px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-81ce5bb flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"81ce5bb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p>As expected, CoreNLP and gemma3 pick up on the most unique gold words. Interestingly, the small and large gemma variants had the fewest unique gold words &#8211; even less than OpenNLP.<\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-85e041a flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"85e041a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-medium\">Experiment 3: Two Pass <\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4fdc3a4 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"4fdc3a4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p>In the previous experiment, we saw that gemma3 listed ~85 \u201cgold\u201d words that no other model did. CoreNLP uniquely listed ~115 \u201cgold\u201d words. We wondered if a better overall result could be achieved by having the two models actually work together to improve their output. Ideally, an LLM could catch some of these \u201cadditional\u201d names (increasing recall) and make some minor changes to CoreNLP\u2019s output (increasing precision).<\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8fd13b7 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-code-highlight\" data-id=\"8fd13b7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"code-highlight.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"prismjs-default copy-to-clipboard \">\n\t\t\t<pre data-line=\"\" class=\"highlight-height language-javascript \">\n\t\t\t\t<code readonly=\"true\" class=\"language-javascript\">\n\t\t\t\t\t<xmp>(LLM Request)\r\n{\r\n  \u201ctext\u201d: \u201cThe 38th NAACP Image Awards televised live on FOX in Hollywood, California, hosted by LL Cool J.\u201d\r\n  \u201corganizations\u201d: [\u201cFOX in\u201d],\r\n  \u201clocations\u201d: [\u201cHollywood, California\u201d]\r\n}\r\n\r\n(LLM Response)\r\n{\r\n  \u201cpeople\u201d: [\u201cLL Cool J\u201d]\r\n  \u201corganizations\u201d: [\u201cFOX\u201d, \u201cNAACP\u201d],\r\n  \u201clocations\u201d: [\u201cHollywood, California\u201d]\r\n}\r\n\r\n<\/xmp>\n\t\t\t\t<\/code>\n\t\t\t<\/pre>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-053160c flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"053160c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p>Our modified Lucille ETL pipeline used only two models. First, CoreNLP extracted entity names, as usual. Then, we used gemma3 again, but with a new system prompt and a different PromptOllama configuration. Now, gemma3 was instructed to \u201cedit\u201d the results from CoreNLP as needed. The stage\u2019s configuration ensured the request included the source text <em>and<\/em> the people, organization, and location names extracted by CoreNLP. (This was the only pipeline where an LLM was provided results from a previous model.) Together, the models had the following scores:<\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f9cb6d5 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\" data-id=\"f9cb6d5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img width=\"653\" height=\"384\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp3.png\" class=\"attachment-large size-large wp-image-30242\" alt=\"\" srcset=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp3.png 653w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp3-300x176.png 300w\" sizes=\"(max-width: 653px) 100vw, 653px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-037e81b flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"037e81b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p>Unfortunately, this approach was actually less performant, with an F1 score lower than CoreNLP or gemma3 operating individually. Instead of finding a way to include those \u201cunique\u201d gold words, it looks like the LLM had an inclination to delete the names output by CoreNLP. (In a later chart, you\u2019ll see the total number of words output in this \u201ctwo pass\u201d pipeline is very similar to the number output by gemma3 alone.) While there are certainly a variety of ways to tweak the pipeline, it seemed we weren\u2019t going to obtain the results we were looking for with this approach.<\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e779735 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"e779735\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-medium\">Experiment 4: Alternate Model<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c6f4d76 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"c6f4d76\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p>Lastly, we wanted to measure the performance of an alternate LLM. We created a pipeline similar to the first experiment, but PromptOllama used deepseek-r1 instead of gemma3. deepseek-r1 is a \u201creasoning\u201d model, which could potentially yield different results. The variant we used, deepseek-r1:14b, had 14 billion parameters, and was roughly 9 GB in size. This made it slightly larger than gemma3:12b, the \u201clarge\u201d model used in the first pipeline.<\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-77fbeaf flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\" data-id=\"77fbeaf\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img width=\"446\" height=\"274\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp4.png\" class=\"attachment-large size-large wp-image-30244\" alt=\"\" srcset=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp4.png 446w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp4-300x184.png 300w\" sizes=\"(max-width: 446px) 100vw, 446px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2ef2fd8 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\" data-id=\"2ef2fd8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" width=\"448\" height=\"275\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp4_gold.png\" class=\"attachment-large size-large wp-image-30243\" alt=\"\" srcset=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp4_gold.png 448w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp4_gold-300x184.png 300w\" sizes=\"(max-width: 448px) 100vw, 448px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9cd80a4 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"9cd80a4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p>The \u201cunique\u201d count only considered these three models, so the results aren\u2019t directly comparable to the same chart from Experiment 2.<\/p><p>We can see that deepseek-r1\u2019s performance was roughly in line with gemma3\u2019s performance from earlier, with an F1 score of roughly 0.7.<\/p><p>Again, we noticed that CoreNLP and the LLM were each picking up on many gold words that the other models weren\u2019t. We still wanted to find a way to capture as many gold words as possible. So, instead of running another pipeline, we decided to just calculate the scores associated with combining the outputs of every model:<\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6378376 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\" data-id=\"6378376\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" width=\"936\" height=\"490\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp4_agg.png\" class=\"attachment-large size-large wp-image-30245\" alt=\"\" srcset=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp4_agg.png 936w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp4_agg-300x157.png 300w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/exp4_agg-768x402.png 768w\" sizes=\"(max-width: 936px) 100vw, 936px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a76aeff flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"a76aeff\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p>As you could imagine, we got higher recall at the cost of reduced precision. We picked up on more of the gold words, but less of the words listed were actually gold words from the original dataset. Interestingly, the F1 score remained roughly the same, despite pronounced shifts in precision and recall.<\/p><p>If you\u2019re looking to run enhanced searches on your documents, a higher recall will help ensure you don\u2019t miss out on any names. But, if you\u2019re looking to run aggregations or facets on the extracted entities, these extra non-gold entries could undermine the quality of your insights.<\/p><p>Again, we noticed that CoreNLP and the LLM were each picking up on many gold words that the other models weren\u2019t. We still wanted to find a way to capture as many gold words as possible. So, instead of running another pipeline, we decided to just calculate the scores associated with combining the outputs of every model:<\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8908134 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"8908134\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-medium\">Pulling The Data Together<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-78106a9 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"78106a9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p>Lastly, here are some higher level results comparing all of the models.<\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9cdbd70 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\" data-id=\"9cdbd70\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" width=\"775\" height=\"484\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_f1.png\" class=\"attachment-large size-large wp-image-30248\" alt=\"\" srcset=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_f1.png 775w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_f1-300x187.png 300w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_f1-768x480.png 768w\" sizes=\"(max-width: 775px) 100vw, 775px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1ec62f8 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\" data-id=\"1ec62f8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" width=\"790\" height=\"531\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_nowords.png\" class=\"attachment-large size-large wp-image-30247\" alt=\"\" srcset=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_nowords.png 790w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_nowords-300x202.png 300w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_nowords-768x516.png 768w\" sizes=\"(max-width: 790px) 100vw, 790px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f0aea71 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\" data-id=\"f0aea71\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" width=\"745\" height=\"459\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_meanlatency.png\" class=\"attachment-large size-large wp-image-30246\" alt=\"\" srcset=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_meanlatency.png 745w, https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/summary_meanlatency-300x185.png 300w\" sizes=\"(max-width: 745px) 100vw, 745px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">CPU \/ GPU: Apple M1 Pro   RAM: 16 GB.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-112f0be flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"112f0be\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-large\">Conclusion<\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b54fdc5 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-text-editor\" data-id=\"b54fdc5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\r\n\t\t\t\t\t\t<p>Overall, it looks like traditional NLP models are still valuable, even as LLMs are frequently touted as the solution to all of life\u2019s problems. CoreNLP generally led the way with the highest F1 scores and only a fraction of the LLMs\u2019 high latency. But some of the LLMs we tested were still very strong alternatives. They had high F1 scores and picked up on some words that CoreNLP didn\u2019t. OpenNLP did underperform, but we were using its pretrained models. Training a custom model with OpenNLP\u2019s architecture could yield improved results.<\/p><p>Though CoreNLP outperformed, LLMs could still play a vital role in many entity extraction solutions, as they are extremely versatile and require minimal setup. If you don\u2019t have the time to find a training dataset, cleanse it, and then train and evaluate a model, an LLM is certainly a viable option. Additionally, an LLM could handle data in a variety of languages without any additional configuration or training needed. If our data was in multiple languages, we would have had to completely overhaul our pipeline to support this data.<\/p><p>As such, any entity extraction solution you build should be tailored to your use case. While you can\u2019t really go wrong with a traditional model, you may want to consider integrating an LLM into your process. Are your documents in multiple languages? Do they have very long pieces of text? How many documents do you have? How much compute is available to you? You\u2019ll have to take a holistic approach to designing your solution.<\/p><p>Based on our findings, even in a world filled with LLMs, it looks like traditional models still have a place in addressing classic NLP problems.<\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-fe57f53\" data-id=\"fe57f53\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap\">\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-82d929d\" data-id=\"82d929d\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-44d87a7 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"44d87a7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"title-h6 elementor-heading-title elementor-size-small\">Share Post<\/div>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5eb80f3 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-thegem-social-sharing\" data-id=\"5eb80f3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"thegem-social-sharing.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\n        <div id=\"thegem-custom-69e4421abcc898400\" class=\"thegem-te-socials-sharing socials-sharing--left socials-sharing--simple socials-sharing--tiny\">\n            <div class=\"socials-sharing\">\n                <ul>\n\t\t\t\t\t                        <li><a class=\"socials-item facebook\" target=\"_blank\" href=\"https:\/\/www.facebook.com\/sharer\/sharer.php?u=https%3A%2F%2Fkmwllc.com%2Findex.php%2Fwp-json%2Fwp%2Fv2%2Fposts%2F30125\" title=\"Facebook\"><i class=\"socials-item-icon facebook\"><\/i><\/a><\/li>\n\t\t\t\t\t\n\t\t\t\t\t                        <li><a class=\"socials-item twitter\" target=\"_blank\" href=\"https:\/\/twitter.com\/intent\/tweet?text=&#038;url=https%3A%2F%2Fkmwllc.com%2Findex.php%2Fwp-json%2Fwp%2Fv2%2Fposts%2F30125\" title=\"Twitter\"><i class=\"socials-item-icon twitter\"><\/i><\/a><\/li>\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t                        <li><a class=\"socials-item linkedin\" target=\"_blank\" href=\"https:\/\/www.linkedin.com\/shareArticle?mini=true&#038;url=https%3A%2F%2Fkmwllc.com%2Findex.php%2Fwp-json%2Fwp%2Fv2%2Fposts%2F30125&#038;title=&amp;summary=\" title=\"LinkedIn\"><i class=\"socials-item-icon linkedin\"><\/i><\/a><\/li>\n\t\t\t\t\t\n\t\t\t\t\t                        <li><a class=\"socials-item reddit\" target=\"_blank\" href=\"https:\/\/www.reddit.com\/submit?url=https%3A%2F%2Fkmwllc.com%2Findex.php%2Fwp-json%2Fwp%2Fv2%2Fposts%2F30125&#038;title=\" title=\"Reddit\"><i class=\"socials-item-icon reddit\"><\/i><\/a><\/li>\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t                        <li><a class=\"socials-item threads\" target=\"_blank\" href=\"https:\/\/www.threads.net\/intent\/post?text=https%3A%2F%2Fkmwllc.com%2Findex.php%2Fwp-json%2Fwp%2Fv2%2Fposts%2F30125\" title=\"Threads\"><i class=\"socials-item-icon threads\"><\/i><\/a><\/li>\n\t\t\t\t\t                <\/ul>\n            <\/div>\n        <\/div>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-aa98076 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\" data-id=\"aa98076\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"title-h6 elementor-heading-title elementor-size-small\">More From the KMW Blog<\/div>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b1d7288 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-thegem-bloglist\" data-id=\"b1d7288\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;thegem_elementor_preset&quot;:&quot;compact-tiny-2&quot;,&quot;source&quot;:[&quot;posts&quot;],&quot;exclude_blog_posts_type&quot;:&quot;current&quot;,&quot;query_type&quot;:&quot;post&quot;,&quot;order_by&quot;:&quot;default&quot;,&quot;order&quot;:&quot;default&quot;,&quot;items_per_page&quot;:8}\" data-widget_type=\"thegem-bloglist.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t<div class=\"bloglist blog clearfix  blog-style-compact-tiny-2   \" data-page=\"1\" data-paged=\"1\" data-next-page=\"2\" data-pages-count=\"3\" data-load-more-action=\"thegem_bloglist_load_more\">\n\t\t\t\r\n<article id=\"post-30279\" class=\"post-item clearfix post-30279 post type-post status-publish format-standard has-post-thumbnail category-elasticsearch category-lucene category-performance\">\r\n\t\t\t<div class=\"gem-compact-tiny-left\">\r\n\t\t\t<div class=\"gem-news-item-image\">\r\n\t\t\t\t<a href=\"https:\/\/kmwllc.com\/index.php\/2026\/01\/10\/the-mystery-of-elasticsearch-8-17-query-performance-degradation\/\"><img loading=\"lazy\" width=\"144\" height=\"144\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2026\/01\/blog_elasticperftest_900x1200-thegem-news-carousel.png\" class=\"img-responsive wp-post-image\" alt=\"blog_elasticperftest_900x1200\" \/><\/a>\r\n\t\t\t<\/div>\r\n\t\t<\/div>\r\n\t\r\n\t<div class=\"gem-compact-tiny-right\">\r\n\t\t<div class=\"gem-compact-item-content\">\r\n\t\t\t<div class=\"tiny-post-title gem-news-item-title text-body-tiny\"><a class=\"reverse-link-color \" href=\"https:\/\/kmwllc.com\/index.php\/2026\/01\/10\/the-mystery-of-elasticsearch-8-17-query-performance-degradation\/\" rel=\"bookmark\">The Mystery of Elasticsearch 8.17 Query Performance Degradation<\/a><\/div>\t\t<\/div>\r\n\t\t<div class=\"post-meta\">\r\n\t\t\t<div class=\"entry-meta clearfix text-body-tiny\">\r\n\t\t\t\t<div class=\"post-meta-left gem-news-item-date\">\r\n\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-author tiny-post-author\">By Henry Caldwell<\/span><br>\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-date tiny-post-date\">January 10, 2026<\/span>\t\t\t\t<\/div>\r\n\t\t\t\t<div class=\"post-meta-right\">\r\n\t\t\t\t\t\t\t\t\t<\/div>\r\n\t\t\t<\/div><!-- .entry-meta -->\r\n\t\t<\/div>\r\n\r\n\t<\/div>\r\n<\/article><!-- #post-30279 -->\r\n\r\n<article id=\"post-30155\" class=\"post-item clearfix post-30155 post type-post status-publish format-standard has-post-thumbnail category-ai\">\r\n\t\t\t<div class=\"gem-compact-tiny-left\">\r\n\t\t\t<div class=\"gem-news-item-image\">\r\n\t\t\t\t<a href=\"https:\/\/kmwllc.com\/index.php\/2025\/05\/20\/mcp-in-llm-apps-overkill-or-integral\/\"><img loading=\"lazy\" width=\"144\" height=\"144\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/05\/blog_mcp_1200x900_min-thegem-news-carousel.png\" class=\"img-responsive wp-post-image\" alt=\"blog_mcp_1200x900_min\" \/><\/a>\r\n\t\t\t<\/div>\r\n\t\t<\/div>\r\n\t\r\n\t<div class=\"gem-compact-tiny-right\">\r\n\t\t<div class=\"gem-compact-item-content\">\r\n\t\t\t<div class=\"tiny-post-title gem-news-item-title text-body-tiny\"><a class=\"reverse-link-color \" href=\"https:\/\/kmwllc.com\/index.php\/2025\/05\/20\/mcp-in-llm-apps-overkill-or-integral\/\" rel=\"bookmark\">MCP in LLM Apps: Overkill or Integral?<\/a><\/div>\t\t<\/div>\r\n\t\t<div class=\"post-meta\">\r\n\t\t\t<div class=\"entry-meta clearfix text-body-tiny\">\r\n\t\t\t\t<div class=\"post-meta-left gem-news-item-date\">\r\n\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-author tiny-post-author\">By Kevin Butler<\/span><br>\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-date tiny-post-date\">May 20, 2025<\/span>\t\t\t\t<\/div>\r\n\t\t\t\t<div class=\"post-meta-right\">\r\n\t\t\t\t\t\t\t\t\t<\/div>\r\n\t\t\t<\/div><!-- .entry-meta -->\r\n\t\t<\/div>\r\n\r\n\t<\/div>\r\n<\/article><!-- #post-30155 -->\r\n\r\n<article id=\"post-29895\" class=\"post-item clearfix post-29895 post type-post status-publish format-standard has-post-thumbnail category-ai category-opensearch category-relevancy category-search category-solr category-vector-search\">\r\n\t\t\t<div class=\"gem-compact-tiny-left\">\r\n\t\t\t<div class=\"gem-news-item-image\">\r\n\t\t\t\t<a href=\"https:\/\/kmwllc.com\/index.php\/2024\/06\/23\/rag-question-answering-system-for-solr-and-opensearch\/\"><img loading=\"lazy\" width=\"144\" height=\"144\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2024\/06\/blog_rag-thegem-news-carousel.png\" class=\"img-responsive wp-post-image\" alt=\"blog_rag\" \/><\/a>\r\n\t\t\t<\/div>\r\n\t\t<\/div>\r\n\t\r\n\t<div class=\"gem-compact-tiny-right\">\r\n\t\t<div class=\"gem-compact-item-content\">\r\n\t\t\t<div class=\"tiny-post-title gem-news-item-title text-body-tiny\"><a class=\"reverse-link-color \" href=\"https:\/\/kmwllc.com\/index.php\/2024\/06\/23\/rag-question-answering-system-for-solr-and-opensearch\/\" rel=\"bookmark\">RAG Question Answering System for Solr and OpenSearch\u00a0<\/a><\/div>\t\t<\/div>\r\n\t\t<div class=\"post-meta\">\r\n\t\t\t<div class=\"entry-meta clearfix text-body-tiny\">\r\n\t\t\t\t<div class=\"post-meta-left gem-news-item-date\">\r\n\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-author tiny-post-author\">By Akul Sethi<\/span><br>\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-date tiny-post-date\">June 23, 2024<\/span>\t\t\t\t<\/div>\r\n\t\t\t\t<div class=\"post-meta-right\">\r\n\t\t\t\t\t\t\t\t\t<\/div>\r\n\t\t\t<\/div><!-- .entry-meta -->\r\n\t\t<\/div>\r\n\r\n\t<\/div>\r\n<\/article><!-- #post-29895 -->\r\n\r\n<article id=\"post-29639\" class=\"post-item clearfix post-29639 post type-post status-publish format-standard has-post-thumbnail category-lucene category-opensearch category-performance category-search\">\r\n\t\t\t<div class=\"gem-compact-tiny-left\">\r\n\t\t\t<div class=\"gem-news-item-image\">\r\n\t\t\t\t<a href=\"https:\/\/kmwllc.com\/index.php\/2024\/05\/30\/duplicate-terms-aggregation-plug-in-for-opensearch\/\"><img loading=\"lazy\" width=\"144\" height=\"144\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2024\/05\/blog_opensearch-agg1200x900-min-thegem-news-carousel.png\" class=\"img-responsive wp-post-image\" alt=\"blog_opensearch-agg1200x900-min\" \/><\/a>\r\n\t\t\t<\/div>\r\n\t\t<\/div>\r\n\t\r\n\t<div class=\"gem-compact-tiny-right\">\r\n\t\t<div class=\"gem-compact-item-content\">\r\n\t\t\t<div class=\"tiny-post-title gem-news-item-title text-body-tiny\"><a class=\"reverse-link-color \" href=\"https:\/\/kmwllc.com\/index.php\/2024\/05\/30\/duplicate-terms-aggregation-plug-in-for-opensearch\/\" rel=\"bookmark\">Duplicate Terms Aggregation Plug-in for OpenSearch<\/a><\/div>\t\t<\/div>\r\n\t\t<div class=\"post-meta\">\r\n\t\t\t<div class=\"entry-meta clearfix text-body-tiny\">\r\n\t\t\t\t<div class=\"post-meta-left gem-news-item-date\">\r\n\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-author tiny-post-author\">By Abijit Rangesh<\/span><br>\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-date tiny-post-date\">May 30, 2024<\/span>\t\t\t\t<\/div>\r\n\t\t\t\t<div class=\"post-meta-right\">\r\n\t\t\t\t\t\t\t\t\t<\/div>\r\n\t\t\t<\/div><!-- .entry-meta -->\r\n\t\t<\/div>\r\n\r\n\t<\/div>\r\n<\/article><!-- #post-29639 -->\r\n\r\n<article id=\"post-28464\" class=\"post-item clearfix post-28464 post type-post status-publish format-standard has-post-thumbnail category-ai category-opensearch category-search category-vector-search\">\r\n\t\t\t<div class=\"gem-compact-tiny-left\">\r\n\t\t\t<div class=\"gem-news-item-image\">\r\n\t\t\t\t<a href=\"https:\/\/kmwllc.com\/index.php\/2023\/03\/29\/building-vector-search-on-opensearch\/\"><img loading=\"lazy\" width=\"144\" height=\"144\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2024\/05\/blog_vectorSearch_1200x900-min-thegem-news-carousel.png\" class=\"img-responsive wp-post-image\" alt=\"blog_vectorSearch_1200x900-min\" \/><\/a>\r\n\t\t\t<\/div>\r\n\t\t<\/div>\r\n\t\r\n\t<div class=\"gem-compact-tiny-right\">\r\n\t\t<div class=\"gem-compact-item-content\">\r\n\t\t\t<div class=\"tiny-post-title gem-news-item-title text-body-tiny\"><a class=\"reverse-link-color \" href=\"https:\/\/kmwllc.com\/index.php\/2023\/03\/29\/building-vector-search-on-opensearch\/\" rel=\"bookmark\">Building A Vector Search Application on OpenSearch<\/a><\/div>\t\t<\/div>\r\n\t\t<div class=\"post-meta\">\r\n\t\t\t<div class=\"entry-meta clearfix text-body-tiny\">\r\n\t\t\t\t<div class=\"post-meta-left gem-news-item-date\">\r\n\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-author tiny-post-author\">By Jake Horban<\/span><br>\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-date tiny-post-date\">March 29, 2023<\/span>\t\t\t\t<\/div>\r\n\t\t\t\t<div class=\"post-meta-right\">\r\n\t\t\t\t\t\t\t\t\t<\/div>\r\n\t\t\t<\/div><!-- .entry-meta -->\r\n\t\t<\/div>\r\n\r\n\t<\/div>\r\n<\/article><!-- #post-28464 -->\r\n\r\n<article id=\"post-28075\" class=\"post-item clearfix post-28075 post type-post status-publish format-standard has-post-thumbnail category-elasticsearch category-search category-solr\">\r\n\t\t\t<div class=\"gem-compact-tiny-left\">\r\n\t\t\t<div class=\"gem-news-item-image\">\r\n\t\t\t\t<a href=\"https:\/\/kmwllc.com\/index.php\/2022\/12\/17\/ingesting-solr-logs-with-the-elk-stack\/\"><img loading=\"lazy\" width=\"144\" height=\"144\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2022\/12\/blog_LogAnalysisElk_min-thegem-news-carousel.png\" class=\"img-responsive wp-post-image\" alt=\"blog_LogAnalysisElk_min\" \/><\/a>\r\n\t\t\t<\/div>\r\n\t\t<\/div>\r\n\t\r\n\t<div class=\"gem-compact-tiny-right\">\r\n\t\t<div class=\"gem-compact-item-content\">\r\n\t\t\t<div class=\"tiny-post-title gem-news-item-title text-body-tiny\"><a class=\"reverse-link-color \" href=\"https:\/\/kmwllc.com\/index.php\/2022\/12\/17\/ingesting-solr-logs-with-the-elk-stack\/\" rel=\"bookmark\">Ingesting Solr Logs with the ELK Stack<\/a><\/div>\t\t<\/div>\r\n\t\t<div class=\"post-meta\">\r\n\t\t\t<div class=\"entry-meta clearfix text-body-tiny\">\r\n\t\t\t\t<div class=\"post-meta-left gem-news-item-date\">\r\n\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-author tiny-post-author\">By Kira Traynor<\/span><br>\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-date tiny-post-date\">December 17, 2022<\/span>\t\t\t\t<\/div>\r\n\t\t\t\t<div class=\"post-meta-right\">\r\n\t\t\t\t\t\t\t\t\t<\/div>\r\n\t\t\t<\/div><!-- .entry-meta -->\r\n\t\t<\/div>\r\n\r\n\t<\/div>\r\n<\/article><!-- #post-28075 -->\r\n\r\n<article id=\"post-27467\" class=\"post-item clearfix post-27467 post type-post status-publish format-standard has-post-thumbnail category-search category-solr\">\r\n\t\t\t<div class=\"gem-compact-tiny-left\">\r\n\t\t\t<div class=\"gem-news-item-image\">\r\n\t\t\t\t<a href=\"https:\/\/kmwllc.com\/index.php\/2022\/11\/17\/solrs-query-elevation-component-now-supports-filter-exclusions\/\"><img loading=\"lazy\" width=\"144\" height=\"144\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2022\/11\/blog_QEC1200x900-thegem-news-carousel.png\" class=\"img-responsive wp-post-image\" alt=\"blog_QEC1200x900\" \/><\/a>\r\n\t\t\t<\/div>\r\n\t\t<\/div>\r\n\t\r\n\t<div class=\"gem-compact-tiny-right\">\r\n\t\t<div class=\"gem-compact-item-content\">\r\n\t\t\t<div class=\"tiny-post-title gem-news-item-title text-body-tiny\"><a class=\"reverse-link-color \" href=\"https:\/\/kmwllc.com\/index.php\/2022\/11\/17\/solrs-query-elevation-component-now-supports-filter-exclusions\/\" rel=\"bookmark\">Solr&#8217;s query elevation component now supports filter exclusions<\/a><\/div>\t\t<\/div>\r\n\t\t<div class=\"post-meta\">\r\n\t\t\t<div class=\"entry-meta clearfix text-body-tiny\">\r\n\t\t\t\t<div class=\"post-meta-left gem-news-item-date\">\r\n\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-author tiny-post-author\">By Rudi Seitz<\/span><br>\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-date tiny-post-date\">November 17, 2022<\/span>\t\t\t\t<\/div>\r\n\t\t\t\t<div class=\"post-meta-right\">\r\n\t\t\t\t\t\t\t\t\t<\/div>\r\n\t\t\t<\/div><!-- .entry-meta -->\r\n\t\t<\/div>\r\n\r\n\t<\/div>\r\n<\/article><!-- #post-27467 -->\r\n\r\n<article id=\"post-26659\" class=\"post-item clearfix post-26659 post type-post status-publish format-standard has-post-thumbnail category-search\">\r\n\t\t\t<div class=\"gem-compact-tiny-left\">\r\n\t\t\t<div class=\"gem-news-item-image\">\r\n\t\t\t\t<a href=\"https:\/\/kmwllc.com\/index.php\/2022\/09\/30\/the-kmw-search-audit\/\"><img loading=\"lazy\" width=\"144\" height=\"144\" src=\"https:\/\/kmwllc.com\/wp-content\/uploads\/2022\/09\/blog_KMWSearchAudit1200x900-thegem-news-carousel.png\" class=\"img-responsive wp-post-image\" alt=\"blog_KMWSearchAudit1200x900\" \/><\/a>\r\n\t\t\t<\/div>\r\n\t\t<\/div>\r\n\t\r\n\t<div class=\"gem-compact-tiny-right\">\r\n\t\t<div class=\"gem-compact-item-content\">\r\n\t\t\t<div class=\"tiny-post-title gem-news-item-title text-body-tiny\"><a class=\"reverse-link-color \" href=\"https:\/\/kmwllc.com\/index.php\/2022\/09\/30\/the-kmw-search-audit\/\" rel=\"bookmark\">The KMW Search Audit<\/a><\/div>\t\t<\/div>\r\n\t\t<div class=\"post-meta\">\r\n\t\t\t<div class=\"entry-meta clearfix text-body-tiny\">\r\n\t\t\t\t<div class=\"post-meta-left gem-news-item-date\">\r\n\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-author tiny-post-author\">By Brian Nauheimer<\/span><br>\t\t\t\t\t<span\r\n\t\t\t\t\t\t\tclass=\"post-meta-date tiny-post-date\">September 30, 2022<\/span>\t\t\t\t<\/div>\r\n\t\t\t\t<div class=\"post-meta-right\">\r\n\t\t\t\t\t\t\t\t\t<\/div>\r\n\t\t\t<\/div><!-- .entry-meta -->\r\n\t\t<\/div>\r\n\r\n\t<\/div>\r\n<\/article><!-- #post-26659 -->\r\n\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div><\/div>\r\n\t\t<\/section>\r\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-0ee5fe2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"0ee5fe2\" data-element_type=\"section\" data-e-type=\"section\">\r\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-thegem\"><div class=\"elementor-row\">\r\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-499f8b9\" data-id=\"499f8b9\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-91a2eda flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-post-navigation\" data-id=\"91a2eda\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"post-navigation.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-post-navigation\" role=\"navigation\" aria-label=\"Post Navigation\">\n\t\t\t<div class=\"elementor-post-navigation__prev elementor-post-navigation__link\">\n\t\t\t\t<a href=\"https:\/\/kmwllc.com\/index.php\/2022\/07\/02\/search-engine-upgrade\/\" rel=\"prev\"><span class=\"elementor-post-navigation__link__prev\"><span class=\"post-navigation__prev--label\">Previous Post<\/span><span class=\"post-navigation__prev--title\">Search Engine Upgrade<\/span><\/span><\/a>\t\t\t<\/div>\n\t\t\t\t\t\t<div class=\"elementor-post-navigation__next elementor-post-navigation__link\">\n\t\t\t\t<a href=\"https:\/\/kmwllc.com\/index.php\/2022\/11\/17\/solrs-query-elevation-component-now-supports-filter-exclusions\/\" rel=\"next\"><span class=\"elementor-post-navigation__link__next\"><span class=\"post-navigation__next--label\">Next Post<\/span><span class=\"post-navigation__next--title\">Solr&#8217;s query elevation component now supports filter exclusions<\/span><\/span><\/a>\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div><\/div>\r\n\t\t<\/section>\r\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Comparing the Effectiveness of Entity Extraction between NLP and LLMs<\/p>\n","protected":false},"author":14,"featured_media":30252,"comment_status":"closed","ping_status":"open","sticky":false,"template":"single-fullwidth.php","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[35,5,47,36,1],"tags":[],"class_list":{"0":"post-30125","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-ai","8":"category-performance","9":"category-relevancy","10":"category-search","11":"category-uncategorized"},"aioseo_notices":[],"post_meta_fields":{"_wp_page_template":["single-fullwidth.php"],"_edit_lock":["1768227838:7"],"_edit_last":["1"],"_aioseo_title":[null],"_aioseo_description":[null],"_aioseo_keywords":["a:0:{}"],"_aioseo_og_title":[null],"_aioseo_og_description":[null],"_aioseo_og_article_section":[""],"_aioseo_og_article_tags":["a:0:{}"],"_aioseo_twitter_title":[null],"_aioseo_twitter_description":[null],"thegem_post_item_gallery_images":[""],"_customize_sidebars":["yes"],"thegem_post_general_item_data":["a:26:{s:20:\"post_layout_settings\";s:6:\"custom\";s:18:\"post_layout_source\";s:7:\"default\";s:21:\"post_builder_template\";s:1:\"0\";s:26:\"show_featured_posts_slider\";i:0;s:21:\"show_featured_content\";s:7:\"default\";s:10:\"video_type\";s:7:\"youtube\";s:5:\"video\";s:0:\"\";s:18:\"video_aspect_ratio\";s:0:\"\";s:20:\"video_play_on_mobile\";s:0:\"\";s:13:\"video_overlay\";s:0:\"\";s:12:\"video_poster\";s:0:\"\";s:11:\"video_start\";s:16:\"open_in_lightbox\";s:10:\"quote_text\";s:0:\"\";s:12:\"quote_author\";s:0:\"\";s:16:\"quote_background\";s:0:\"\";s:18:\"quote_author_color\";s:0:\"\";s:5:\"audio\";s:0:\"\";s:7:\"gallery\";i:0;s:18:\"gallery_autoscroll\";i:0;s:9:\"highlight\";i:0;s:14:\"highlight_type\";s:7:\"squared\";s:15:\"highlight_style\";s:7:\"default\";s:31:\"highlight_title_left_background\";s:9:\"#00BCD4FF\";s:26:\"highlight_title_left_color\";s:9:\"#FFFFFFFF\";s:32:\"highlight_title_right_background\";s:9:\"#00BCD4FF\";s:27:\"highlight_title_right_color\";s:9:\"#FFFFFFFF\";}"],"thegem_show_featured_posts_slider":["0"],"thegem_post_page_elements_data":["a:12:{s:13:\"post_elements\";s:6:\"custom\";s:11:\"show_author\";i:0;s:16:\"blog_hide_author\";i:1;s:14:\"blog_hide_date\";i:1;s:26:\"blog_hide_date_in_blog_cat\";i:1;s:20:\"blog_hide_categories\";i:1;s:14:\"blog_hide_tags\";i:1;s:18:\"blog_hide_comments\";i:1;s:15:\"blog_hide_likes\";i:1;s:20:\"blog_hide_navigation\";i:1;s:17:\"blog_hide_socials\";i:1;s:17:\"blog_hide_realted\";i:0;}"],"thegem_popups_data":["a:2:{s:20:\"popups_layout_source\";s:7:\"default\";s:12:\"thegemPopups\";a:0:{}}"],"thegem_page_data":["a:200:{s:10:\"title_show\";s:7:\"enabled\";s:11:\"title_style\";s:1:\"2\";s:14:\"title_template\";s:5:\"27835\";s:23:\"title_use_page_settings\";i:1;s:12:\"title_xlarge\";i:0;s:18:\"title_rich_content\";i:0;s:13:\"title_content\";s:0:\"\";s:21:\"title_background_type\";s:5:\"color\";s:22:\"title_background_image\";s:0:\"\";s:29:\"title_background_image_repeat\";i:0;s:27:\"title_background_position_x\";s:6:\"center\";s:27:\"title_background_position_y\";s:3:\"top\";s:21:\"title_background_size\";s:5:\"cover\";s:28:\"title_background_image_color\";s:0:\"\";s:30:\"title_background_image_overlay\";s:0:\"\";s:30:\"title_background_gradient_type\";s:6:\"linear\";s:31:\"title_background_gradient_angle\";i:90;s:34:\"title_background_gradient_position\";s:13:\"center center\";s:38:\"title_background_gradient_point1_color\";s:9:\"#00BCD4BF\";s:41:\"title_background_gradient_point1_position\";i:0;s:38:\"title_background_gradient_point2_color\";s:9:\"#354093BF\";s:41:\"title_background_gradient_point2_position\";i:100;s:23:\"title_background_effect\";s:6:\"normal\";s:36:\"title_background_ken_burns_direction\";s:7:\"zoom_in\";s:43:\"title_background_ken_burns_transition_speed\";i:15000;s:37:\"title_background_video_play_on_mobile\";i:0;s:22:\"title_background_color\";s:7:\"#333144\";s:27:\"title_background_video_type\";s:0:\"\";s:22:\"title_background_video\";s:0:\"\";s:35:\"title_background_video_aspect_ratio\";s:0:\"\";s:36:\"title_background_video_overlay_color\";s:0:\"\";s:38:\"title_background_video_overlay_opacity\";s:0:\"\";s:29:\"title_background_video_poster\";s:0:\"\";s:19:\"title_menu_on_video\";s:0:\"\";s:16:\"title_text_color\";s:7:\"#ffffff\";s:24:\"title_excerpt_text_color\";s:7:\"#ffffff\";s:13:\"title_excerpt\";s:0:\"\";s:18:\"title_hide_excerpt\";i:0;s:17:\"title_title_width\";i:0;s:19:\"title_excerpt_width\";i:0;s:22:\"title_font_preset_html\";s:0:\"\";s:23:\"title_font_preset_style\";s:0:\"\";s:24:\"title_font_preset_weight\";s:0:\"\";s:27:\"title_font_preset_transform\";s:0:\"\";s:30:\"title_excerpt_font_preset_html\";s:0:\"\";s:31:\"title_excerpt_font_preset_style\";s:0:\"\";s:32:\"title_excerpt_font_preset_weight\";s:0:\"\";s:35:\"title_excerpt_font_preset_transform\";s:0:\"\";s:17:\"title_padding_top\";i:80;s:24:\"title_padding_top_tablet\";i:80;s:24:\"title_padding_top_mobile\";i:80;s:20:\"title_padding_bottom\";i:80;s:27:\"title_padding_bottom_tablet\";i:80;s:27:\"title_padding_bottom_mobile\";i:80;s:18:\"title_padding_left\";i:0;s:25:\"title_padding_left_tablet\";i:0;s:25:\"title_padding_left_mobile\";i:0;s:19:\"title_padding_right\";i:0;s:26:\"title_padding_right_tablet\";i:0;s:26:\"title_padding_right_mobile\";i:0;s:16:\"title_top_margin\";s:0:\"\";s:23:\"title_top_margin_tablet\";i:0;s:23:\"title_top_margin_mobile\";i:0;s:24:\"title_excerpt_top_margin\";i:18;s:31:\"title_excerpt_top_margin_tablet\";i:18;s:31:\"title_excerpt_top_margin_mobile\";i:18;s:17:\"title_breadcrumbs\";i:1;s:15:\"title_alignment\";s:0:\"\";s:15:\"title_icon_pack\";s:7:\"elegant\";s:10:\"title_icon\";s:0:\"\";s:16:\"title_icon_color\";s:0:\"\";s:18:\"title_icon_color_2\";s:0:\"\";s:27:\"title_icon_background_color\";s:0:\"\";s:16:\"title_icon_shape\";s:6:\"circle\";s:23:\"title_icon_border_color\";s:0:\"\";s:15:\"title_icon_size\";s:5:\"large\";s:16:\"title_icon_style\";s:0:\"\";s:18:\"title_icon_opacity\";d:0;s:25:\"breadcrumbs_default_color\";s:0:\"\";s:24:\"breadcrumbs_active_color\";s:0:\"\";s:23:\"breadcrumbs_hover_color\";s:0:\"\";s:27:\"title_breadcrumbs_alignment\";s:6:\"center\";s:18:\"header_transparent\";i:0;s:25:\"header_transparent_mobile\";i:0;s:14:\"header_opacity\";i:50;s:22:\"header_menu_logo_light\";i:0;s:20:\"disable_fixed_header\";i:0;s:23:\"sticky_header_on_mobile\";s:0:\"\";s:20:\"header_hide_top_area\";s:7:\"default\";s:27:\"header_hide_top_area_tablet\";s:7:\"default\";s:27:\"header_hide_top_area_mobile\";s:7:\"default\";s:9:\"menu_show\";s:7:\"default\";s:12:\"menu_options\";s:7:\"default\";s:18:\"header_custom_menu\";i:0;s:27:\"header_top_area_transparent\";i:0;s:23:\"header_top_area_opacity\";i:50;s:16:\"top_area_options\";s:7:\"default\";s:13:\"header_source\";s:7:\"default\";s:14:\"header_builder\";s:1:\"0\";s:29:\"header_builder_sticky_desktop\";i:0;s:28:\"header_builder_sticky_mobile\";i:0;s:34:\"header_builder_sticky_hide_desktop\";i:0;s:33:\"header_builder_sticky_hide_mobile\";i:1;s:21:\"header_builder_sticky\";s:1:\"0\";s:29:\"header_builder_sticky_opacity\";i:80;s:26:\"header_builder_light_color\";s:7:\"#FFFFFF\";s:32:\"header_builder_light_color_hover\";s:7:\"#00bcd4\";s:20:\"main_background_type\";s:5:\"color\";s:21:\"main_background_color\";s:7:\"#ffffff\";s:21:\"main_background_image\";s:60:\"https:\/\/kmwllc.com\/wp-content\/uploads\/2022\/11\/Artboard-3.png\";s:28:\"main_background_image_repeat\";s:1:\"1\";s:26:\"main_background_position_x\";s:4:\"left\";s:26:\"main_background_position_y\";s:3:\"top\";s:20:\"main_background_size\";s:4:\"auto\";s:27:\"main_background_image_color\";s:0:\"\";s:29:\"main_background_image_overlay\";s:0:\"\";s:29:\"main_background_gradient_type\";s:6:\"linear\";s:30:\"main_background_gradient_angle\";s:2:\"90\";s:33:\"main_background_gradient_position\";s:0:\"\";s:37:\"main_background_gradient_point1_color\";s:9:\"#E9ECDAFF\";s:40:\"main_background_gradient_point1_position\";s:1:\"0\";s:37:\"main_background_gradient_point2_color\";s:9:\"#D5F6FAFF\";s:40:\"main_background_gradient_point2_position\";s:3:\"100\";s:23:\"main_background_pattern\";s:0:\"\";s:19:\"content_padding_top\";i:50;s:26:\"content_padding_top_tablet\";i:30;s:26:\"content_padding_top_mobile\";i:20;s:22:\"content_padding_bottom\";i:50;s:29:\"content_padding_bottom_tablet\";i:30;s:29:\"content_padding_bottom_mobile\";i:20;s:20:\"content_area_options\";s:6:\"custom\";s:18:\"footer_custom_show\";s:7:\"default\";s:13:\"footer_custom\";s:5:\"28054\";s:19:\"footer_hide_default\";s:7:\"default\";s:23:\"footer_hide_widget_area\";s:7:\"default\";s:16:\"effects_disabled\";i:0;s:17:\"effects_one_pager\";i:0;s:23:\"effects_parallax_footer\";i:0;s:24:\"effects_no_bottom_margin\";i:0;s:21:\"effects_no_top_margin\";i:0;s:19:\"redirect_to_subpage\";i:0;s:19:\"effects_hide_header\";s:7:\"default\";s:19:\"effects_hide_footer\";s:7:\"default\";s:21:\"effects_page_scroller\";i:0;s:28:\"effects_page_scroller_mobile\";i:0;s:26:\"effects_page_scroller_type\";s:8:\"advanced\";s:22:\"fullpage_disabled_dots\";i:0;s:19:\"fullpage_style_dots\";s:7:\"outline\";s:31:\"fullpage_disabled_tooltips_dots\";i:0;s:25:\"fullpage_fixed_background\";b:0;s:26:\"fullpage_enable_continuous\";i:0;s:24:\"fullpage_disabled_mobile\";i:0;s:22:\"fullpage_scroll_effect\";s:6:\"normal\";s:21:\"enable_page_preloader\";s:7:\"default\";s:14:\"slideshow_type\";s:0:\"\";s:19:\"slideshow_slideshow\";s:0:\"\";s:21:\"slideshow_layerslider\";s:0:\"\";s:19:\"slideshow_revslider\";s:0:\"\";s:19:\"slideshow_preloader\";i:0;s:12:\"sidebar_show\";s:7:\"default\";s:16:\"sidebar_position\";s:5:\"right\";s:14:\"sidebar_sticky\";i:0;s:24:\"product_header_separator\";i:0;s:23:\"page_layout_breadcrumbs\";s:7:\"default\";s:37:\"page_layout_breadcrumbs_default_color\";s:9:\"#99A9B5FF\";s:36:\"page_layout_breadcrumbs_active_color\";s:9:\"#3C3950FF\";s:35:\"page_layout_breadcrumbs_hover_color\";s:9:\"#3C3950FF\";s:33:\"page_layout_breadcrumbs_alignment\";s:4:\"left\";s:38:\"page_layout_breadcrumbs_bottom_spacing\";s:1:\"0\";s:37:\"page_layout_breadcrumbs_shop_category\";i:0;s:26:\"delay_js_execution_desktop\";i:0;s:13:\"disable_cache\";i:0;s:23:\"custom_fields_item_data\";N;s:31:\"title_breadcrumbs_shop_category\";s:1:\"0\";s:20:\"title_padding_locked\";s:0:\"\";s:27:\"title_padding_tablet_locked\";s:0:\"\";s:27:\"title_padding_mobile_locked\";s:0:\"\";s:24:\"title_background_pattern\";s:0:\"\";s:30:\"title_background_video_overlay\";s:0:\"\";s:16:\"title_icon__pack\";s:0:\"\";s:21:\"title_icon_shape_show\";s:0:\"\";s:25:\"footer_widget_woocommerce\";s:1:\"1\";s:19:\"fullpage_dots_color\";s:0:\"\";s:33:\"fullpage_dots_tolltips_text_color\";s:0:\"\";s:31:\"fullpage_dots_tolltips_bg_color\";s:0:\"\";s:25:\"delay_js_execution_mobile\";s:1:\"0\";s:18:\"custom_fields_data\";s:0:\"\";s:14:\"page_item_data\";a:3:{s:20:\"page_layout_settings\";s:7:\"default\";s:18:\"page_layout_source\";s:7:\"default\";s:21:\"page_builder_template\";s:0:\"\";}s:19:\"portfolio_item_data\";a:35:{s:25:\"portfolio_layout_settings\";s:7:\"default\";s:23:\"portfolio_layout_source\";s:7:\"default\";s:26:\"portfolio_builder_template\";s:0:\"\";s:8:\"back_url\";s:0:\"\";s:9:\"highlight\";s:0:\"\";s:14:\"highlight_type\";s:0:\"\";s:14:\"overview_title\";s:0:\"\";s:16:\"overview_summary\";s:0:\"\";s:12:\"project_link\";s:0:\"\";s:12:\"project_text\";s:0:\"\";s:9:\"fullwidth\";s:0:\"\";s:19:\"project_button_show\";s:0:\"\";s:20:\"grid_appearance_type\";s:14:\"featured_image\";s:30:\"grid_appearance_featured_image\";s:0:\"\";s:19:\"grid_appearance_gif\";s:0:\"\";s:25:\"grid_appearance_gif_start\";s:8:\"autoplay\";s:26:\"grid_appearance_gif_poster\";s:0:\"\";s:27:\"grid_appearance_gif_preload\";s:0:\"\";s:21:\"grid_appearance_video\";s:0:\"\";s:26:\"grid_appearance_video_type\";s:0:\"\";s:34:\"grid_appearance_video_aspect_ratio\";s:0:\"\";s:36:\"grid_appearance_video_play_on_mobile\";s:0:\"\";s:29:\"grid_appearance_video_overlay\";s:0:\"\";s:28:\"grid_appearance_video_poster\";s:0:\"\";s:27:\"grid_appearance_video_start\";s:8:\"autoplay\";s:34:\"grid_appearance_gallery_autoscroll\";s:1:\"1\";s:40:\"grid_appearance_gallery_autoscroll_speed\";s:4:\"3000\";s:30:\"grid_appearance_image_behavior\";s:12:\"link_to_page\";s:28:\"grid_appearance_gif_behavior\";s:12:\"link_to_page\";s:30:\"grid_appearance_video_behavior\";s:12:\"link_to_page\";s:32:\"grid_appearance_gallery_behavior\";s:12:\"link_to_page\";s:31:\"grid_appearance_behavior_target\";s:5:\"_self\";s:36:\"grid_appearance_behavior_custom_link\";s:0:\"\";s:43:\"grid_appearance_behavior_custom_link_target\";s:5:\"_self\";s:30:\"grid_appearance_lightbox_image\";s:0:\"\";}s:23:\"portfolio_elements_data\";a:7:{s:23:\"portfolio_page_elements\";s:7:\"default\";s:19:\"portfolio_hide_date\";s:0:\"\";s:19:\"portfolio_hide_sets\";s:0:\"\";s:20:\"portfolio_hide_likes\";s:0:\"\";s:22:\"portfolio_hide_socials\";s:0:\"\";s:29:\"portfolio_hide_top_navigation\";s:0:\"\";s:32:\"portfolio_hide_bottom_navigation\";s:0:\"\";}s:17:\"product_item_data\";a:117:{s:9:\"highlight\";s:0:\"\";s:14:\"highlight_type\";s:7:\"squared\";s:28:\"thegem_product_disable_hover\";s:1:\"0\";s:10:\"size_guide\";s:7:\"default\";s:16:\"size_guide_image\";s:0:\"\";s:15:\"size_guide_text\";s:10:\"Size guide\";s:23:\"product_layout_settings\";s:7:\"default\";s:21:\"product_layout_source\";s:7:\"default\";s:24:\"product_builder_template\";s:0:\"\";s:19:\"product_page_layout\";s:7:\"default\";s:25:\"product_page_layout_style\";s:15:\"horizontal_tabs\";s:28:\"product_page_layout_centered\";s:1:\"0\";s:39:\"product_page_layout_centered_top_margin\";s:2:\"42\";s:34:\"product_page_layout_centered_boxed\";s:1:\"0\";s:40:\"product_page_layout_centered_boxed_color\";s:0:\"\";s:29:\"product_page_layout_fullwidth\";s:1:\"0\";s:26:\"product_page_layout_sticky\";s:1:\"0\";s:33:\"product_page_layout_sticky_offset\";s:1:\"0\";s:28:\"product_page_skeleton_loader\";s:1:\"0\";s:30:\"product_page_layout_background\";s:0:\"\";s:30:\"product_page_layout_title_area\";s:8:\"disabled\";s:29:\"product_page_ajax_add_to_cart\";s:1:\"1\";s:31:\"product_page_desc_review_source\";s:17:\"extra_description\";s:31:\"product_page_desc_review_layout\";s:4:\"tabs\";s:42:\"product_page_desc_review_layout_tabs_style\";s:10:\"horizontal\";s:46:\"product_page_desc_review_layout_tabs_alignment\";s:4:\"left\";s:44:\"product_page_desc_review_layout_acc_position\";s:13:\"below_gallery\";s:65:\"product_page_desc_review_layout_one_by_one_description_background\";s:9:\"#F4F6F7FF\";s:69:\"product_page_desc_review_layout_one_by_one_additional_info_background\";s:9:\"#FFFFFFFF\";s:61:\"product_page_desc_review_layout_one_by_one_reviews_background\";s:9:\"#F4F6F7FF\";s:36:\"product_page_desc_review_description\";s:1:\"1\";s:42:\"product_page_desc_review_description_title\";s:11:\"Description\";s:40:\"product_page_desc_review_additional_info\";s:1:\"1\";s:46:\"product_page_desc_review_additional_info_title\";s:15:\"Additional Info\";s:32:\"product_page_desc_review_reviews\";s:1:\"1\";s:38:\"product_page_desc_review_reviews_title\";s:7:\"Reviews\";s:36:\"product_page_button_add_to_cart_text\";s:11:\"Add to Cart\";s:36:\"product_page_button_add_to_cart_icon\";s:4:\"f1e7\";s:41:\"product_page_button_add_to_cart_icon_pack\";s:8:\"material\";s:45:\"product_page_button_add_to_cart_icon_position\";s:4:\"left\";s:40:\"product_page_button_add_to_wishlist_icon\";s:4:\"f37b\";s:45:\"product_page_button_add_to_wishlist_icon_pack\";s:8:\"material\";s:42:\"product_page_button_added_to_wishlist_icon\";s:4:\"f377\";s:47:\"product_page_button_added_to_wishlist_icon_pack\";s:8:\"material\";s:41:\"product_page_button_clear_attributes_text\";s:15:\"Clear selection\";s:31:\"product_page_elements_prev_next\";s:1:\"1\";s:38:\"product_page_elements_preview_on_hover\";s:1:\"1\";s:34:\"product_page_elements_back_to_shop\";s:1:\"1\";s:39:\"product_page_elements_back_to_shop_link\";s:9:\"main_shop\";s:50:\"product_page_elements_back_to_shop_link_custom_url\";s:0:\"\";s:27:\"product_page_elements_title\";s:1:\"1\";s:32:\"product_page_elements_title_html\";s:0:\"\";s:32:\"product_page_elements_attributes\";s:1:\"0\";s:37:\"product_page_elements_attributes_data\";s:0:\"\";s:29:\"product_page_elements_reviews\";s:1:\"1\";s:34:\"product_page_elements_reviews_text\";s:16:\"customer reviews\";s:27:\"product_page_elements_price\";s:1:\"1\";s:41:\"product_page_elements_price_strikethrough\";s:1:\"1\";s:33:\"product_page_elements_description\";s:1:\"1\";s:34:\"product_page_elements_stock_amount\";s:1:\"1\";s:39:\"product_page_elements_stock_amount_text\";s:17:\"Products in stock\";s:32:\"product_page_elements_size_guide\";s:1:\"1\";s:25:\"product_page_elements_sku\";s:1:\"1\";s:31:\"product_page_elements_sku_title\";s:3:\"SKU\";s:32:\"product_page_elements_categories\";s:1:\"1\";s:38:\"product_page_elements_categories_title\";s:10:\"Categories\";s:26:\"product_page_elements_tags\";s:1:\"1\";s:32:\"product_page_elements_tags_title\";s:4:\"Tags\";s:27:\"product_page_elements_share\";s:1:\"1\";s:33:\"product_page_elements_share_title\";s:5:\"Share\";s:36:\"product_page_elements_share_facebook\";s:1:\"1\";s:35:\"product_page_elements_share_twitter\";s:1:\"1\";s:37:\"product_page_elements_share_pinterest\";s:1:\"1\";s:34:\"product_page_elements_share_tumblr\";s:1:\"1\";s:36:\"product_page_elements_share_linkedin\";s:1:\"1\";s:34:\"product_page_elements_share_reddit\";s:1:\"1\";s:28:\"product_page_elements_upsell\";s:1:\"1\";s:34:\"product_page_elements_upsell_title\";s:17:\"You may also like\";s:44:\"product_page_elements_upsell_title_alignment\";s:4:\"left\";s:34:\"product_page_elements_upsell_items\";s:2:\"-1\";s:44:\"product_page_elements_upsell_columns_desktop\";s:2:\"4x\";s:43:\"product_page_elements_upsell_columns_tablet\";s:2:\"3x\";s:43:\"product_page_elements_upsell_columns_mobile\";s:2:\"2x\";s:40:\"product_page_elements_upsell_columns_100\";s:1:\"5\";s:29:\"product_page_elements_related\";s:1:\"1\";s:35:\"product_page_elements_related_title\";s:16:\"Related Products\";s:45:\"product_page_elements_related_title_alignment\";s:4:\"left\";s:35:\"product_page_elements_related_items\";s:2:\"-1\";s:45:\"product_page_elements_related_columns_desktop\";s:2:\"4x\";s:44:\"product_page_elements_related_columns_tablet\";s:2:\"3x\";s:44:\"product_page_elements_related_columns_mobile\";s:2:\"2x\";s:41:\"product_page_elements_related_columns_100\";s:1:\"5\";s:28:\"product_page_additional_tabs\";s:7:\"default\";s:33:\"product_page_additional_tabs_data\";s:0:\"\";s:15:\"product_gallery\";s:7:\"enabled\";s:20:\"product_gallery_type\";s:10:\"horizontal\";s:31:\"product_gallery_thumb_on_mobile\";s:1:\"0\";s:31:\"product_gallery_column_position\";s:4:\"left\";s:30:\"product_gallery_thumb_position\";s:4:\"left\";s:28:\"product_gallery_column_width\";s:2:\"50\";s:26:\"product_gallery_show_image\";s:5:\"hover\";s:27:\"product_gallery_image_ratio\";s:0:\"\";s:31:\"product_gallery_grid_image_size\";s:7:\"default\";s:32:\"product_gallery_grid_image_ratio\";s:0:\"\";s:20:\"product_gallery_zoom\";s:1:\"1\";s:24:\"product_gallery_lightbox\";s:1:\"1\";s:22:\"product_gallery_labels\";s:1:\"1\";s:26:\"product_gallery_label_sale\";s:1:\"1\";s:25:\"product_gallery_label_new\";s:1:\"1\";s:31:\"product_gallery_label_out_stock\";s:1:\"1\";s:27:\"product_gallery_auto_height\";s:1:\"1\";s:30:\"product_gallery_elements_color\";s:0:\"\";s:28:\"product_gallery_grid_columns\";s:2:\"1x\";s:25:\"product_gallery_grid_gaps\";s:2:\"42\";s:30:\"product_gallery_grid_gaps_hide\";s:1:\"0\";s:31:\"product_gallery_grid_top_margin\";s:1:\"0\";s:30:\"product_gallery_video_autoplay\";s:1:\"0\";}s:25:\"product_archive_item_data\";a:5:{s:29:\"product_archive_layout_source\";s:7:\"default\";s:32:\"product_archive_builder_template\";s:0:\"\";s:24:\"products_layout_settings\";s:7:\"default\";s:22:\"products_layout_source\";s:7:\"default\";s:25:\"products_builder_template\";s:0:\"\";}s:22:\"blog_archive_item_data\";a:3:{s:28:\"blog_archive_layout_settings\";s:7:\"default\";s:26:\"blog_archive_layout_source\";s:7:\"default\";s:29:\"blog_archive_builder_template\";s:0:\"\";}s:24:\"custom_archive_item_data\";a:6:{s:15:\"layout_settings\";s:7:\"default\";s:13:\"layout_source\";s:7:\"default\";s:16:\"builder_template\";s:0:\"\";s:21:\"posts_layout_settings\";s:7:\"default\";s:19:\"posts_layout_source\";s:7:\"default\";s:22:\"posts_builder_template\";s:0:\"\";}s:21:\"custom_post_item_data\";a:26:{s:15:\"layout_settings\";s:7:\"default\";s:13:\"layout_source\";s:7:\"default\";s:16:\"builder_template\";s:0:\"\";s:10:\"video_type\";s:0:\"\";s:18:\"video_aspect_ratio\";s:0:\"\";s:5:\"video\";s:0:\"\";s:20:\"video_play_on_mobile\";s:0:\"\";s:13:\"video_overlay\";s:0:\"\";s:12:\"video_poster\";s:0:\"\";s:11:\"video_start\";s:16:\"open_in_lightbox\";s:5:\"audio\";s:0:\"\";s:10:\"quote_text\";s:0:\"\";s:12:\"quote_author\";s:0:\"\";s:16:\"quote_background\";s:0:\"\";s:18:\"quote_author_color\";s:0:\"\";s:7:\"gallery\";s:0:\"\";s:18:\"gallery_autoscroll\";s:4:\"1500\";s:26:\"show_featured_posts_slider\";s:0:\"\";s:21:\"show_featured_content\";s:0:\"\";s:9:\"highlight\";s:0:\"\";s:14:\"highlight_type\";s:0:\"\";s:15:\"highlight_style\";s:0:\"\";s:31:\"highlight_title_left_background\";s:9:\"#00BCD4FF\";s:32:\"highlight_title_right_background\";s:9:\"#00BCD4FF\";s:26:\"highlight_title_left_color\";s:9:\"#FFFFFFFF\";s:27:\"highlight_title_right_color\";s:9:\"#FFFFFFFF\";}s:24:\"options_current_contents\";N;s:16:\"options_modified\";N;s:34:\"options_outside_parameter_modified\";b:0;s:22:\"options_saved_contents\";N;s:8:\"settings\";a:3:{s:5:\"theme\";s:5:\"light\";s:24:\"background_image_gallery\";a:0:{}s:21:\"colorpicker_favorites\";a:1:{s:7:\"default\";a:0:{}}}}"],"_elementor_edit_mode":["builder"],"_elementor_template_type":["wp-post"],"_elementor_version":["3.25.10"],"_elementor_pro_version":["3.22.1"],"_zilla_likes":["0"],"_elementor_data":["[{\"id\":\"5260c7c\",\"elType\":\"section\",\"settings\":{\"structure\":\"30\"},\"elements\":[{\"id\":\"f697649\",\"elType\":\"column\",\"settings\":{\"_column_size\":33,\"_inline_size\":65,\"thegem_column_breakpoints_list\":[],\"_inline_size_tablet\":100},\"elements\":[{\"id\":\"2598a50\",\"elType\":\"widget\",\"settings\":{\"icon_list\":[{\"selected_icon\":{\"value\":\"fas fa-calendar\",\"library\":\"fa-solid\"},\"_id\":\"91a0f52\",\"custom_date_format\":\"F j, Y\",\"custom_time_format\":\"g:i a\",\"link\":\"\",\"show_icon\":\"none\"}],\"icon_typography_typography\":\"custom\",\"__globals__\":{\"text_color\":\"globals\\\/colors?id=primary\"}},\"elements\":[],\"widgetType\":\"post-info\"},{\"id\":\"4829f73\",\"elType\":\"widget\",\"settings\":{\"title\":\"Add Your Heading Text Here\",\"header_size\":\"div\",\"thegem_heading_style\":\"styled-subtitle\",\"__dynamic__\":{\"title\":\"[elementor-tag id=\\\"7b6ad14\\\" name=\\\"thegem-post-excerpt\\\" settings=\\\"%7B%7D\\\"]\"}},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"3906aea\",\"elType\":\"widget\",\"settings\":{\"source\":\"custom\",\"author_avatar\":{\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/Jacob.jpg\",\"id\":30208,\"alt\":\"\",\"source\":\"library\",\"size\":\"\"},\"author_name\":\"Jacob Squatrito\",\"author_name_tag\":\"div\",\"author_bio\":\"Search & AI Engineer at KMW Technology\",\"link_text\":\"All Posts\",\"image_vertical_align\":\"top\",\"image_border\":\"yes\",\"image_border_width\":{\"unit\":\"px\",\"size\":3,\"sizes\":[]},\"image_border_radius\":{\"unit\":\"px\",\"size\":100,\"sizes\":[]},\"name_gap\":{\"unit\":\"px\",\"size\":0,\"sizes\":[]},\"__globals__\":{\"image_border_color\":\"globals\\\/colors?id=428f277\",\"bio_color\":\"globals\\\/colors?id=secondary\"},\"_padding\":{\"unit\":\"em\",\"top\":\"2\",\"right\":\"2\",\"bottom\":\"2\",\"left\":\"2\",\"isLinked\":true},\"_padding_tablet\":{\"unit\":\"em\",\"top\":\"\",\"right\":\"\",\"bottom\":\"\",\"left\":\"\",\"isLinked\":true},\"_padding_mobile\":{\"unit\":\"em\",\"top\":\"\",\"right\":\"\",\"bottom\":\"\",\"left\":\"\",\"isLinked\":true}},\"elements\":[],\"widgetType\":\"author-box\"},{\"id\":\"7a1c601\",\"elType\":\"widget\",\"settings\":{\"text\":\"Divider\",\"weight\":{\"unit\":\"px\",\"size\":3,\"sizes\":[]},\"gap\":{\"unit\":\"px\",\"size\":0,\"sizes\":[]},\"__globals__\":{\"color\":\"globals\\\/colors?id=428f277\"}},\"elements\":[],\"widgetType\":\"divider\"},{\"id\":\"54998f4\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p><span style=\\\"font-weight: 400;\\\">Entities are really important for search, but what\\u2019s the best way to do it?<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">The ability to analyze a piece of text and identify the key entities within it can have lots of practical search uses like improving search relevancy; enabling faceting and filtering classifying documents and even obfuscation of sensitive data.<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">As a natural language problem, entity extraction is not new but it\\u2019s always been tricky to do well. There are lots of traditional NLP models for entity extraction and recently LLMs have shown promising abilities too. For search applications we are typically trying to balance excellent entity extraction along with operational needs like running fast and not consuming too many resources. So what is the best way to do entity extraction for modern search applications?<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">In this blog post we are going to compare traditional NLP models with LLMs to see how they measure up. We'll focus on extracting the names of people, organizations, or locations within a body of text. We\\u2019ll discuss ways entity extraction can improve the search experience, analyze the performance of traditional models and large language models, run a few experiments, and conclude with a review of our findings<\\\/span><\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"8717084\",\"elType\":\"widget\",\"settings\":{\"title\":\"A Simple Example\",\"size\":\"large\",\"header_size\":\"h1\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"5831465\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p><span style=\\\"font-weight: 400;\\\">Imagine we have four documents we want to index:<\\\/span><\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"2e5a0c6\",\"elType\":\"widget\",\"settings\":{\"language\":\"json\",\"code\":\"{\\r\\n  \\u201cid\\u201d: 1\\r\\n  \\u201ctext\\u201d: \\u201cWill, are you going to the store today?\\u201d\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 2\\r\\n  \\u201ctext\\u201d: \\u201cWill you go to the store today?\\u201d\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 3\\r\\n  \\u201ctext\\u201d: \\u201cI hope you will join us.\\u201d\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 4\\r\\n  \\u201ctext\\u201d: \\u201cIs Hope going to be joining us?\\u201d\\r\\n}\\r\\n\\r\\n\\r\\n\",\"line_numbers\":\"\"},\"elements\":[],\"widgetType\":\"code-highlight\"},{\"id\":\"a9743f6\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p><span style=\\\"font-weight: 400;\\\">Let\\u2019s say you want to search for documents referencing a certain person. In some cases, you might get away with just searching for their name, as-is, in <\\\/span><span style=\\\"font-weight: 400;\\\">text<\\\/span><span style=\\\"font-weight: 400;\\\">. But, when you search for names like Will or Hope, your search engine will likely return documents that use these words in a different context. As you can see, \\u201cwill\\u201d is found in documents 1, 2, and 3, but only document 1 actually references someone named Will. We run into a similar issue when searching for \\u201chope\\u201d as well.\\u00a0<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">Having some irrelevant documents returned is, of course, not ideal. But in practice, the problem may be more than just a minor nuisance. Terms like Will and Hope could be used more often as English words rather than names. Documents that naturally use the word \\u201cwill\\u201d multiple times might score higher than documents mentioning a person named Will. Making matters even worse, the text might reference Will with pronouns instead of stating his name repeatedly, further decreasing the document\\u2019s search score. <\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">So\\u2026 just running a search on <\\\/span><span style=\\\"font-weight: 400;\\\">\\u201ctext\\u201d<\\\/span><span style=\\\"font-weight: 400;\\\"> won\\u2019t always suffice. To avoid manually pruning through your search results to remove the irrelevant documents, you\\u2019ll want to enrich your content before indexing.\\u00a0<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">With entity extraction, we can enrich our documents like so:<\\\/span><\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"3a9e936\",\"elType\":\"widget\",\"settings\":{\"language\":\"json\",\"code\":\"{\\r\\n  \\u201cid\\u201d: 1\\r\\n  \\u201ctext\\u201d: \\u201cWill, are you going to the store today?\\u201d\\r\\n  \\u201cpeople\\u201d: [\\u201cWill\\u201d]\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 2\\r\\n  \\u201ctext\\u201d: \\u201cWill you go to the store today?\\u201d\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 3\\r\\n  \\u201ctext\\u201d: \\u201cI hope you will join us.\\u201d\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 4\\r\\n  \\u201ctext\\u201d: \\u201cIs Hope going to be joining us?\\u201d\\r\\n  \\u201cpeople\\u201d: [\\u201cHope\\u201d]\\r\\n}\\r\\n\",\"line_numbers\":\"\"},\"elements\":[],\"widgetType\":\"code-highlight\"},{\"id\":\"c95a3aa\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p><span style=\\\"font-weight: 400;\\\">If we extract entities from the text, we can attach lists of names mentioned to the document. Now, if we want to search for documents that reference somebody named Will, we would search on <\\\/span><span style=\\\"font-weight: 400;\\\">\\u201cpeople\\u201d<\\\/span><span style=\\\"font-weight: 400;\\\"> instead of <\\\/span><span style=\\\"font-weight: 400;\\\">\\u201ctext\\u201d<\\\/span><span style=\\\"font-weight: 400;\\\">. This allows us to ensure documents like 2 and 3 don\\u2019t hinder our search process or contaminate our results. This is definitely a simple use case - different use cases will certainly have unique challenges to overcome.<\\\/span><\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"4722b30\",\"elType\":\"widget\",\"settings\":{\"title\":\"Our Entity Extraction Approach\",\"size\":\"large\",\"header_size\":\"h1\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"9370169\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p><span style=\\\"font-weight: 400;\\\">There exist many statistical\\\/rules-based models for entity extraction and named entity recognition. For our testing, we used Apache OpenNLP\\u2019s <\\\/span><a href=\\\"https:\\\/\\\/opennlp.sourceforge.net\\\/models-1.5\\\/\\\"><span style=\\\"font-weight: 400;\\\">pretrained models<\\\/span><\\\/a><span style=\\\"font-weight: 400;\\\"> and Stanford CoreNLP\\u2019s built-in named entity recognition models. But they aren\\u2019t perfect, and with the ever increasing popularity of large language models (LLMs), we thought it would be interesting to see how an LLM performs entity extraction compared against these legacy approaches. <\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">We theorized that an LLM had the potential to extract certain names that more traditional models may be likely to miss. (In particular, we figured an LLM would be more likely to extract a \\u201cnewer\\u201d name that was uncommon when the traditional models were trained.) But, we also believed an LLM could potentially get \\u201cdistracted\\u201d and underperform the traditional models on longer pieces of text.<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">In order to test our theories, we ran four different experiments:<\\\/span><\\\/p><ol><li style=\\\"font-weight: 400;\\\" aria-level=\\\"1\\\"><span style=\\\"font-weight: 400;\\\"><strong>Baseline:<\\\/strong> We started by evaluating the performance of OpenNLP, CoreNLP, and Google\\u2019s <\\\/span><a href=\\\"https:\\\/\\\/ollama.com\\\/library\\\/gemma3\\\"><span style=\\\"font-weight: 400;\\\">gemma3<\\\/span><\\\/a><span style=\\\"font-weight: 400;\\\">, a popular and capable open-source LLM. The default version of the model is about 3 GB in size and has roughly 4 billion parameters, making it suitable for use on modern hardware. For each model, we evaluated its precision, recall, and F1 score.<\\\/span><\\\/li><li style=\\\"font-weight: 400;\\\" aria-level=\\\"1\\\"><span style=\\\"font-weight: 400;\\\"><strong>Number of Parameters:<\\\/strong> We introduced two new variants of gemma3 with a different number of parameters. We discussed how using more \\\/ less parameters appeared to affect the results.<\\\/span><\\\/li><li style=\\\"font-weight: 400;\\\" aria-level=\\\"1\\\"><span style=\\\"font-weight: 400;\\\"><strong>Two Pass<\\\/strong>: Using the two strongest models - CoreNLP and gemma3 - we instructed gemma3 to observe and edit the output of CoreNLP as it saw fit.\\u00a0<\\\/span><\\\/li><li style=\\\"font-weight: 400;\\\" aria-level=\\\"1\\\"><span style=\\\"font-weight: 400;\\\"><strong>Alternate Model:<\\\/strong> We evaluated another popular model, <\\\/span><a href=\\\"https:\\\/\\\/ollama.com\\\/library\\\/deepseek-r1\\\"><span style=\\\"font-weight: 400;\\\">deepseek-r1<\\\/span><\\\/a><span style=\\\"font-weight: 400;\\\">, against OpenNLP and CoreNLP, to see if there were any notable differences.<\\\/span><\\\/li><\\\/ol><p><span style=\\\"font-weight: 400;\\\">For all experiments, we used a publicly available and fully annotated <\\\/span><a href=\\\"https:\\\/\\\/github.com\\\/juand-r\\\/entity-recognition-datasets\\\/blob\\\/master\\\/data\\\/wikigold\\\/CONLL-format\\\/data\\\/wikigold.conll.txt\\\"><span style=\\\"font-weight: 400;\\\">wikigold dataset<\\\/span><\\\/a><span style=\\\"font-weight: 400;\\\">, allowing us to evaluate the models against a source of truth. Across the 140+ wiki articles, there were roughly 3,000 words annotated as a person, organization, or location.\\u00a0<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">As part of our evaluation, we worked with <\\\/span><a href=\\\"https:\\\/\\\/github.com\\\/kmwtechnology\\\/lucille\\\"><span style=\\\"font-weight: 400;\\\">Lucille<\\\/span><\\\/a><span style=\\\"font-weight: 400;\\\">, our open-source Search ETL solution that allowed us to pass text through entity extraction processes and update each document with the output. We created a custom Connector to process the wikigold dataset into Lucille, including the article\\u2019s text as well as lists of the annotated (or \\u201cgold\\u201d) people, organization, and location names.\\u00a0<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">To perform entity extraction, we used three different Lucille Stages. We created a custom Stage to extract people, organization, and location names using OpenNLP. We did the same for CoreNLP as well. To work with an LLM, we used Lucille\\u2019s PromptOllama Stage, which allows you to provide parts (or all) of a document to a compatible LLM for generic enrichment. The model was instructed to read the source text and output a JSON object including the names of people, organizations, and locations mentioned in the document. Lucille then integrated the model\\u2019s JSON response into the document. The models did not have access to the output of other Stages - they only saw the source text.\\u00a0<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">Here\\u2019s an example of what a finalized document looked like, after we normalized the output for evaluation:<\\\/span><\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"bc53fdf\",\"elType\":\"widget\",\"settings\":{\"code\":\"{\\r\\n  \\u201ctext\\u201d: \\u201c010 is the tenth album from Japanese Punk Techno band The Mad Capsule Markets . This album proved to be more commercial and more techno-based than Osc-Dis , with heavily synthesized songs like Introduction 010 and Come . Founding member Kojima Minoru played guitar on Good Day , and Wardanceis cover of a song by UK post punk industrial band Killing Joke . XXX can of This had a different meaning , and most people did n't understand what the song was about . it was later explained that the song was about Cannabis ( ' can of this ' sounding like Cannabis when said faster ) it is uncertain if they were told to change the lyric like they did on P.O.P and HUMANITY . UK Edition came with the OSC-DIS video , and most of the tracks were re-engineered .\\u201d\\r\\n  \\u201copenNLP_people\\u201d: []\\r\\n  \\u201ccoreNLP_people\\u201d: [\\u201ckojima\\u201d, \\u201cminoru\\u201d]\\r\\n  \\u201collama_people\\u201d: []\\r\\n  \\u201cgold_people\\u201d: [\\u201ckojima\\u201d, \\u201cminoru\\u201d]\\r\\n  \\u201copenNLP_organizations\\u201d: [\\u201cuk\\u201d, \\u201ckilling\\u201d, \\u201cjoke\\u201d, \\u201cfounding\\u201d]\\r\\n  \\u201ccoreNLP_organizations\\u201d: []\\r\\n  \\u201collama_organizations\\u201d: [\\u201cthe\\u201d, \\u201cmad\\u201d, \\u201ccapsule\\u201d, \\u201cmarkets\\u201d, \\u201cmeta\\u201d, \\u201ckilling\\u201d, \\u201cjoke\\u201d]\\r\\n  \\u201cgold_organizations\\u201d: [\\u201cthe\\u201d, \\u201ckilling\\u201d, \\u201cmad\\u201d, \\u201cmarkets\\u201d, \\u201ccapsule\\u201d, \\u201cjoke\\u201d]\\r\\n  \\u201copenNLP_locations\\u201d: []\\r\\n  \\u201ccoreNLP_locations\\u201d: [\\u201cuk\\u201d]\\r\\n  \\u201collama_locations\\u201d: [\\u201cuk\\u201d]\\r\\n  \\u201cgold_locations\\u201d: [\\u201cuk\\u201d]\\r\\n}\\r\\n\",\"line_numbers\":\"\"},\"elements\":[],\"widgetType\":\"code-highlight\"},{\"id\":\"b5e3211\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p><span style=\\\"font-weight: 400;\\\">For the purposes of evaluation, we did not index the documents into a search engine. Instead, they were indexed into a CSV, which stored the original text, annotated entities, and output from each model. We then ran a custom script to analyze the models\\u2019 performance. Using the annotated people, organizations, and locations from the wikigold dataset, we were able to compute some key metrics for each model:<\\\/span><\\\/p><ul><li aria-level=\\\"1\\\"><b>Precision<\\\/b><span style=\\\"font-weight: 400;\\\"> - What percentage of the person\\\/organization\\\/location names output by a model were annotated as such in the dataset?<\\\/span><\\\/li><\\\/ul><ul><li aria-level=\\\"1\\\"><b>Recall<\\\/b><span style=\\\"font-weight: 400;\\\"> - What percentage of the annotated person\\\/organization\\\/location names in the dataset were output by the model?<\\\/span><\\\/li><\\\/ul><ul><li style=\\\"font-weight: 400;\\\" aria-level=\\\"1\\\"><b>F1 <\\\/b><span style=\\\"font-weight: 400;\\\">- The \\u201charmonic mean\\u201d of precision and recall. Considered a solid overall indicator of a model\\u2019s performance.\\u00a0<br \\\/><\\\/span><\\\/li><li><b>Unique Gold Words<\\\/b><span style=\\\"font-weight: 400;\\\"> - How many gold names did a model mention that no other model did?<\\\/span><\\\/li><\\\/ul>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"b827d24\",\"elType\":\"widget\",\"settings\":{\"title\":\"ExperIminets &amp; Results\",\"size\":\"large\",\"header_size\":\"h1\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"719512e\",\"elType\":\"widget\",\"settings\":{\"title\":\"Experiment 1: Number of Parameters\",\"size\":\"medium\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"8861a2a\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p><span style=\\\"font-weight: 400;\\\">We began by creating a pipeline that used three different models: OpenNLP, CoreNLP, and gemma3. Each model ran independently of the other, meaning they were not aware of each other\\u2019s output. Here are the precision, recall, and F1 scores for each model:<\\\/span><\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"a33b483\",\"elType\":\"widget\",\"settings\":{\"carousel\":[{\"id\":30214,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1gemma3.png\"},{\"id\":30213,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1opennlp.png\"},{\"id\":30212,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1corenlp.png\"}],\"thumbnail_size\":\"medium_large\",\"slides_to_show\":\"1\"},\"elements\":[],\"widgetType\":\"image-carousel\"},{\"id\":\"0d0ebb4\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p><span style=\\\"font-weight: 400;\\\">Clearly, CoreNLP is the strongest contender here, with gemma3 in a close second. Both had similar F1 scores of about ~0.75. OpenNLP wasn\\u2019t the strongest contender with a lower F1 score. We should also consider the latency associated with running the LLM.<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">On average, gemma3 took about 8 seconds to respond per Document, slowing the pipeline down substantially. (The experiment was run on an Apple M1 Pro with 16 GB of RAM.)<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">We also analyzed the effect of text length on model performance. Here, we considered just the top performing models - CoreNLP and gemma3. We calculated the same metrics (precision, recall, F1) <\\\/span><i><span style=\\\"font-weight: 400;\\\">for each<\\\/span><\\\/i><span style=\\\"font-weight: 400;\\\"> wiki article. There were a few articles with 900+ words that we excluded to avoid skewing the results. We also excluded articles with less than 100 words. Since these documents were very short, they usually just didn\\u2019t reference names of a certain type. As a result, the model scores were primarily either zero or one, which made the results very volatile and difficult to observe:<\\\/span><\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"461ca00\",\"elType\":\"widget\",\"settings\":{\"carousel\":[{\"id\":30218,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_gemma3_f1.png\"},{\"id\":30219,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_corenlp_f1.png\"},{\"id\":30220,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_gemma3_recall.png\"},{\"id\":30221,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_corenlp_recall.png\"},{\"id\":30222,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_gemma3_precision.png\"},{\"id\":30223,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_corenlp_precision.png\"}],\"thumbnail_size\":\"medium_large\",\"slides_to_show\":\"1\"},\"elements\":[],\"widgetType\":\"image-carousel\"},{\"id\":\"4d8ec49\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p><span style=\\\"font-weight: 400;\\\">As expected, the data here was a bit scattered. Since many points overlapped at the top and bottom of each chart, we did include lines of best fit to help visualize the overall trend. However, these lines should be interpreted cautiously, as they all had a <\\\/span><i><span style=\\\"font-weight: 400;\\\">very<\\\/span><\\\/i><span style=\\\"font-weight: 400;\\\"> low R-squared value. In other words, the length of a piece of text shouldn\\u2019t be used to singularly predict the recall, precision, or F1 score you\\u2019ll get from a model.<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\">Again, these charts should be observed with caution, as there were a multitude of factors at play here. But, it does seem reasonable to suggest that there was some sort of relationship between longer text and LLM underperformance. CoreNLP, on the other hand, appears to have been remarkably consistent.<\\\/span><\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"b244a1e\",\"elType\":\"widget\",\"settings\":{\"title\":\"Experiment 2: Number of Parameters\",\"size\":\"medium\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"3ee8fc6\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p><span style=\\\"font-weight: 400;\\\">Our next pipeline included five different models (OpenNLP, CoreNLP, and three variants of gemma3). In addition to the previous pipeline\\u2019s models, we added a smaller variant of gemma3, <\\\/span><a href=\\\"https:\\\/\\\/ollama.com\\\/library\\\/gemma3:1b\\\"><span style=\\\"font-weight: 400;\\\">gemma3:1b<\\\/span><\\\/a><span style=\\\"font-weight: 400;\\\">, and a larger variant of gemma3, <\\\/span><a href=\\\"https:\\\/\\\/ollama.com\\\/library\\\/gemma3:12b\\\"><span style=\\\"font-weight: 400;\\\">gemma3:12b<\\\/span><\\\/a><span style=\\\"font-weight: 400;\\\">. As we mentioned above, <\\\/span><a href=\\\"https:\\\/\\\/ollama.com\\\/library\\\/gemma3:latest\\\"><span style=\\\"font-weight: 400;\\\">gemma3<\\\/span><\\\/a><span style=\\\"font-weight: 400;\\\">, had 4 billion parameters and was a little more than 3 GB in size. The smaller variant, gemma3:1b, had 1 billion parameters and was less than 1 GB in size. The larger variant, <\\\/span><a href=\\\"https:\\\/\\\/ollama.com\\\/library\\\/gemma3:12b\\\"><span style=\\\"font-weight: 400;\\\">gemma3:12b<\\\/span><\\\/a><span style=\\\"font-weight: 400;\\\">, had 12 billion parameters and was about 8 GB in size.<\\\/span><\\\/p><p><span style=\\\"font-weight: 400;\\\"> Here are the results we found:<\\\/span><\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"23914b5\",\"elType\":\"widget\",\"settings\":{\"carousel\":[{\"id\":30232,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_gemma3_12b-1.png\"},{\"id\":30233,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_gemma3-1.png\"},{\"id\":30234,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_gemma3_1b-1.png\"},{\"id\":30228,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_opennlp.png\"},{\"id\":30227,\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_corenlp.png\"}],\"thumbnail_size\":\"medium_large\",\"slides_to_show\":\"1\"},\"elements\":[],\"widgetType\":\"image-carousel\"},{\"id\":\"46b8c90\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p>Again, gemma3 and CoreNLP are the strongest contenders here. The smallest LLM, gemma3:1b, didn\\u2019t do well - not only did it have a poor F1 score, but we found it was actually struggling to follow our instructions. Surprisingly, the largest LLM, gemma3:12b, was actually a bit worse than the medium variant, gemma3. Compared to gemma3, gemma3:12b had a somewhat higher precision but a notably lower recall. It seems that this larger model was a bit too cautious when engaging with the source text.<\\\/p><p id=\\\"ember375\\\" class=\\\"ember-view reader-text-block__paragraph\\\">For this experiment, we also calculated the number of \\u201cgold\\u201d words that were uniquely mentioned by each model. <\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"01e24bd\",\"elType\":\"widget\",\"settings\":{\"image\":{\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_goldwords.png\",\"id\":30235,\"size\":\"\",\"alt\":\"\",\"source\":\"library\"}},\"elements\":[],\"widgetType\":\"image\"},{\"id\":\"81ce5bb\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p>As expected, CoreNLP and gemma3 pick up on the most unique gold words. Interestingly, the small and large gemma variants had the fewest unique gold words - even less than OpenNLP.<\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"85e041a\",\"elType\":\"widget\",\"settings\":{\"title\":\"Experiment 3: Two Pass \",\"size\":\"medium\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"4fdc3a4\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p>In the previous experiment, we saw that gemma3 listed ~85 \\u201cgold\\u201d words that no other model did. CoreNLP uniquely listed ~115 \\u201cgold\\u201d words. We wondered if a better overall result could be achieved by having the two models actually work together to improve their output. Ideally, an LLM could catch some of these \\u201cadditional\\u201d names (increasing recall) and make some minor changes to CoreNLP\\u2019s output (increasing precision).<\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"8fd13b7\",\"elType\":\"widget\",\"settings\":{\"code\":\"(LLM Request)\\r\\n{\\r\\n  \\u201ctext\\u201d: \\u201cThe 38th NAACP Image Awards televised live on FOX in Hollywood, California, hosted by LL Cool J.\\u201d\\r\\n  \\u201corganizations\\u201d: [\\u201cFOX in\\u201d],\\r\\n  \\u201clocations\\u201d: [\\u201cHollywood, California\\u201d]\\r\\n}\\r\\n\\r\\n(LLM Response)\\r\\n{\\r\\n  \\u201cpeople\\u201d: [\\u201cLL Cool J\\u201d]\\r\\n  \\u201corganizations\\u201d: [\\u201cFOX\\u201d, \\u201cNAACP\\u201d],\\r\\n  \\u201clocations\\u201d: [\\u201cHollywood, California\\u201d]\\r\\n}\\r\\n\\r\\n\",\"line_numbers\":\"\"},\"elements\":[],\"widgetType\":\"code-highlight\"},{\"id\":\"053160c\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p>Our modified Lucille ETL pipeline used only two models. First, CoreNLP extracted entity names, as usual. Then, we used gemma3 again, but with a new system prompt and a different PromptOllama configuration. Now, gemma3 was instructed to \\u201cedit\\u201d the results from CoreNLP as needed. The stage\\u2019s configuration ensured the request included the source text <em>and<\\\/em> the people, organization, and location names extracted by CoreNLP. (This was the only pipeline where an LLM was provided results from a previous model.) Together, the models had the following scores:<\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"f9cb6d5\",\"elType\":\"widget\",\"settings\":{\"image\":{\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp3.png\",\"id\":30242,\"size\":\"\",\"alt\":\"\",\"source\":\"library\"}},\"elements\":[],\"widgetType\":\"image\"},{\"id\":\"037e81b\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p>Unfortunately, this approach was actually less performant, with an F1 score lower than CoreNLP or gemma3 operating individually. Instead of finding a way to include those \\u201cunique\\u201d gold words, it looks like the LLM had an inclination to delete the names output by CoreNLP. (In a later chart, you\\u2019ll see the total number of words output in this \\u201ctwo pass\\u201d pipeline is very similar to the number output by gemma3 alone.) While there are certainly a variety of ways to tweak the pipeline, it seemed we weren\\u2019t going to obtain the results we were looking for with this approach.<\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"e779735\",\"elType\":\"widget\",\"settings\":{\"title\":\"Experiment 4: Alternate Model\",\"size\":\"medium\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"c6f4d76\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p>Lastly, we wanted to measure the performance of an alternate LLM. We created a pipeline similar to the first experiment, but PromptOllama used deepseek-r1 instead of gemma3. deepseek-r1 is a \\u201creasoning\\u201d model, which could potentially yield different results. The variant we used, deepseek-r1:14b, had 14 billion parameters, and was roughly 9 GB in size. This made it slightly larger than gemma3:12b, the \\u201clarge\\u201d model used in the first pipeline.<\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"77fbeaf\",\"elType\":\"widget\",\"settings\":{\"image\":{\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4.png\",\"id\":30244,\"size\":\"\",\"alt\":\"\",\"source\":\"library\"}},\"elements\":[],\"widgetType\":\"image\"},{\"id\":\"2ef2fd8\",\"elType\":\"widget\",\"settings\":{\"image\":{\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4_gold.png\",\"id\":30243,\"size\":\"\",\"alt\":\"\",\"source\":\"library\"}},\"elements\":[],\"widgetType\":\"image\"},{\"id\":\"9cd80a4\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p>The \\u201cunique\\u201d count only considered these three models, so the results aren\\u2019t directly comparable to the same chart from Experiment 2.<\\\/p><p>We can see that deepseek-r1\\u2019s performance was roughly in line with gemma3\\u2019s performance from earlier, with an F1 score of roughly 0.7.<\\\/p><p>Again, we noticed that CoreNLP and the LLM were each picking up on many gold words that the other models weren\\u2019t. We still wanted to find a way to capture as many gold words as possible. So, instead of running another pipeline, we decided to just calculate the scores associated with combining the outputs of every model:<\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"6378376\",\"elType\":\"widget\",\"settings\":{\"image\":{\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4_agg.png\",\"id\":30245,\"size\":\"\",\"alt\":\"\",\"source\":\"library\"}},\"elements\":[],\"widgetType\":\"image\"},{\"id\":\"a76aeff\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p>As you could imagine, we got higher recall at the cost of reduced precision. We picked up on more of the gold words, but less of the words listed were actually gold words from the original dataset. Interestingly, the F1 score remained roughly the same, despite pronounced shifts in precision and recall.<\\\/p><p>If you\\u2019re looking to run enhanced searches on your documents, a higher recall will help ensure you don\\u2019t miss out on any names. But, if you\\u2019re looking to run aggregations or facets on the extracted entities, these extra non-gold entries could undermine the quality of your insights.<\\\/p><p>Again, we noticed that CoreNLP and the LLM were each picking up on many gold words that the other models weren\\u2019t. We still wanted to find a way to capture as many gold words as possible. So, instead of running another pipeline, we decided to just calculate the scores associated with combining the outputs of every model:<\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"8908134\",\"elType\":\"widget\",\"settings\":{\"title\":\"Pulling The Data Together\",\"size\":\"medium\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"78106a9\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p>Lastly, here are some higher level results comparing all of the models.<\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"},{\"id\":\"9cdbd70\",\"elType\":\"widget\",\"settings\":{\"image\":{\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_f1.png\",\"id\":30248,\"size\":\"\",\"alt\":\"\",\"source\":\"library\"}},\"elements\":[],\"widgetType\":\"image\"},{\"id\":\"1ec62f8\",\"elType\":\"widget\",\"settings\":{\"image\":{\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_nowords.png\",\"id\":30247,\"size\":\"\",\"alt\":\"\",\"source\":\"library\"}},\"elements\":[],\"widgetType\":\"image\"},{\"id\":\"f0aea71\",\"elType\":\"widget\",\"settings\":{\"image\":{\"url\":\"https:\\\/\\\/kmwllccom.stage.site\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_meanlatency.png\",\"id\":30246,\"size\":\"\",\"alt\":\"\",\"source\":\"library\"},\"caption_source\":\"custom\",\"caption\":\"CPU \\\/ GPU: Apple M1 Pro   RAM: 16 GB.\"},\"elements\":[],\"widgetType\":\"image\"},{\"id\":\"112f0be\",\"elType\":\"widget\",\"settings\":{\"title\":\"Conclusion\",\"size\":\"large\",\"header_size\":\"h1\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"b54fdc5\",\"elType\":\"widget\",\"settings\":{\"editor\":\"<p>Overall, it looks like traditional NLP models are still valuable, even as LLMs are frequently touted as the solution to all of life\\u2019s problems. CoreNLP generally led the way with the highest F1 scores and only a fraction of the LLMs\\u2019 high latency. But some of the LLMs we tested were still very strong alternatives. They had high F1 scores and picked up on some words that CoreNLP didn\\u2019t. OpenNLP did underperform, but we were using its pretrained models. Training a custom model with OpenNLP\\u2019s architecture could yield improved results.<\\\/p><p>Though CoreNLP outperformed, LLMs could still play a vital role in many entity extraction solutions, as they are extremely versatile and require minimal setup. If you don\\u2019t have the time to find a training dataset, cleanse it, and then train and evaluate a model, an LLM is certainly a viable option. Additionally, an LLM could handle data in a variety of languages without any additional configuration or training needed. If our data was in multiple languages, we would have had to completely overhaul our pipeline to support this data.<\\\/p><p>As such, any entity extraction solution you build should be tailored to your use case. While you can\\u2019t really go wrong with a traditional model, you may want to consider integrating an LLM into your process. Are your documents in multiple languages? Do they have very long pieces of text? How many documents do you have? How much compute is available to you? You\\u2019ll have to take a holistic approach to designing your solution.<\\\/p><p>Based on our findings, even in a world filled with LLMs, it looks like traditional models still have a place in addressing classic NLP problems.<\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"}],\"isInner\":false},{\"id\":\"fe57f53\",\"elType\":\"column\",\"settings\":{\"_column_size\":33,\"_inline_size\":10,\"thegem_column_breakpoints_list\":[]},\"elements\":[],\"isInner\":false},{\"id\":\"82d929d\",\"elType\":\"column\",\"settings\":{\"_column_size\":33,\"_inline_size\":24.666,\"thegem_column_breakpoints_list\":[],\"_inline_size_tablet\":100},\"elements\":[{\"id\":\"44d87a7\",\"elType\":\"widget\",\"settings\":{\"title\":\"Share Post\",\"size\":\"small\",\"header_size\":\"div\",\"thegem_heading_style\":\"title-h6\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"5eb80f3\",\"elType\":\"widget\",\"settings\":{\"pinterest\":\"\",\"tumblr\":\"\",\"telegram\":\"\",\"whatsapp\":\"\",\"viber\":\"\",\"xing\":\"\",\"icons_color\":\"#00DEFF\"},\"elements\":[],\"widgetType\":\"thegem-social-sharing\"},{\"id\":\"aa98076\",\"elType\":\"widget\",\"settings\":{\"title\":\"More From the KMW Blog\",\"size\":\"small\",\"header_size\":\"div\",\"thegem_heading_style\":\"title-h6\"},\"elements\":[],\"widgetType\":\"heading\"},{\"id\":\"b1d7288\",\"elType\":\"widget\",\"settings\":{\"thegem_elementor_preset\":\"compact-tiny-2\",\"source\":[\"posts\"],\"show_separator\":\"\",\"show_comments\":\"\",\"readmore_button_text\":\"Read More\",\"loadmore_button_text\":\"Load More\",\"caption_categories_in_text\":\"in \",\"caption_author_by_text\":\"By\",\"source_type\":\"custom\",\"exclude_blog_posts_type\":\"current\",\"pagination_type\":\"numbers\"},\"elements\":[],\"widgetType\":\"thegem-bloglist\"}],\"isInner\":false}],\"isInner\":false},{\"id\":\"0ee5fe2\",\"elType\":\"section\",\"settings\":[],\"elements\":[{\"id\":\"499f8b9\",\"elType\":\"column\",\"settings\":{\"_column_size\":100,\"_inline_size\":null,\"thegem_column_breakpoints_list\":[]},\"elements\":[{\"id\":\"91a2eda\",\"elType\":\"widget\",\"settings\":{\"prev_label\":\"Previous Post\",\"next_label\":\"Next Post\",\"show_borders\":\"\",\"title_typography_typography\":\"custom\",\"title_typography_font_size\":{\"unit\":\"px\",\"size\":14,\"sizes\":[]},\"title_typography_font_weight\":\"700\",\"__globals__\":{\"arrow_color\":\"globals\\\/colors?id=primary\",\"label_color\":\"globals\\\/colors?id=secondary\"},\"arrow\":\"fa fa-caret-left\",\"show_arrow\":\"\"},\"elements\":[],\"widgetType\":\"post-navigation\"}],\"isInner\":false}],\"isInner\":false}]"],"_thumbnail_id":["30252"],"_wp_old_date":["2025-10-07"],"_elementor_controls_usage":["a:13:{s:9:\"post-info\";a:3:{s:5:\"count\";i:1;s:15:\"control_percent\";i:0;s:8:\"controls\";a:2:{s:7:\"content\";a:1:{s:12:\"section_icon\";a:1:{s:9:\"icon_list\";i:1;}}s:5:\"style\";a:1:{s:18:\"section_text_style\";a:1:{s:26:\"icon_typography_typography\";i:1;}}}}s:7:\"heading\";a:3:{s:5:\"count\";i:12;s:15:\"control_percent\";i:1;s:8:\"controls\";a:2:{s:7:\"general\";a:1:{s:11:\"__dynamic__\";a:1:{s:5:\"count\";i:1;}}s:7:\"content\";a:1:{s:13:\"section_title\";a:4:{s:5:\"title\";i:12;s:11:\"header_size\";i:7;s:20:\"thegem_heading_style\";i:3;s:4:\"size\";i:11;}}}}s:10:\"author-box\";a:3:{s:5:\"count\";i:1;s:15:\"control_percent\";i:3;s:8:\"controls\";a:3:{s:7:\"content\";a:1:{s:19:\"section_author_info\";a:5:{s:6:\"source\";i:1;s:13:\"author_avatar\";i:1;s:11:\"author_name\";i:1;s:15:\"author_name_tag\";i:1;s:10:\"author_bio\";i:1;}}s:5:\"style\";a:2:{s:19:\"section_image_style\";a:4:{s:20:\"image_vertical_align\";i:1;s:12:\"image_border\";i:1;s:18:\"image_border_width\";i:1;s:19:\"image_border_radius\";i:1;}s:18:\"section_text_style\";a:1:{s:8:\"name_gap\";i:1;}}s:8:\"advanced\";a:1:{s:14:\"_section_style\";a:3:{s:8:\"_padding\";i:1;s:15:\"_padding_tablet\";i:1;s:15:\"_padding_mobile\";i:1;}}}}s:7:\"divider\";a:3:{s:5:\"count\";i:1;s:15:\"control_percent\";i:0;s:8:\"controls\";a:1:{s:5:\"style\";a:1:{s:21:\"section_divider_style\";a:2:{s:6:\"weight\";i:1;s:3:\"gap\";i:1;}}}}s:11:\"text-editor\";a:3:{s:5:\"count\";i:20;s:15:\"control_percent\";i:0;s:8:\"controls\";a:1:{s:7:\"content\";a:1:{s:14:\"section_editor\";a:1:{s:6:\"editor\";i:20;}}}}s:14:\"code-highlight\";a:3:{s:5:\"count\";i:4;s:15:\"control_percent\";i:0;s:8:\"controls\";a:1:{s:7:\"content\";a:1:{s:15:\"section_content\";a:3:{s:8:\"language\";i:2;s:4:\"code\";i:4;s:12:\"line_numbers\";i:4;}}}}s:14:\"image-carousel\";a:3:{s:5:\"count\";i:3;s:15:\"control_percent\";i:1;s:8:\"controls\";a:1:{s:7:\"content\";a:1:{s:22:\"section_image_carousel\";a:3:{s:8:\"carousel\";i:3;s:14:\"thumbnail_size\";i:3;s:14:\"slides_to_show\";i:3;}}}}s:5:\"image\";a:3:{s:5:\"count\";i:8;s:15:\"control_percent\";i:1;s:8:\"controls\";a:1:{s:7:\"content\";a:1:{s:13:\"section_image\";a:3:{s:5:\"image\";i:8;s:14:\"caption_source\";i:1;s:7:\"caption\";i:1;}}}}s:6:\"column\";a:3:{s:5:\"count\";i:4;s:15:\"control_percent\";i:0;s:8:\"controls\";a:1:{s:6:\"layout\";a:1:{s:6:\"layout\";a:2:{s:12:\"_inline_size\";i:4;s:19:\"_inline_size_tablet\";i:2;}}}}s:21:\"thegem-social-sharing\";a:3:{s:5:\"count\";i:1;s:15:\"control_percent\";i:2;s:8:\"controls\";a:2:{s:7:\"content\";a:1:{s:13:\"section_icons\";a:6:{s:9:\"pinterest\";i:1;s:6:\"tumblr\";i:1;s:8:\"telegram\";i:1;s:8:\"whatsapp\";i:1;s:5:\"viber\";i:1;s:4:\"xing\";i:1;}}s:5:\"style\";a:1:{s:5:\"Style\";a:1:{s:11:\"icons_color\";i:1;}}}}s:15:\"thegem-bloglist\";a:3:{s:5:\"count\";i:1;s:15:\"control_percent\";i:0;s:8:\"controls\";a:1:{s:7:\"content\";a:3:{s:14:\"section_layout\";a:1:{s:23:\"thegem_elementor_preset\";i:1;}s:12:\"section_blog\";a:2:{s:6:\"source\";i:1;s:23:\"exclude_blog_posts_type\";i:1;}s:15:\"section_caption\";a:2:{s:14:\"show_separator\";i:1;s:13:\"show_comments\";i:1;}}}}s:7:\"section\";a:3:{s:5:\"count\";i:2;s:15:\"control_percent\";i:0;s:8:\"controls\";a:1:{s:6:\"layout\";a:1:{s:17:\"section_structure\";a:1:{s:9:\"structure\";i:1;}}}}s:15:\"post-navigation\";a:3:{s:5:\"count\";i:1;s:15:\"control_percent\";i:2;s:8:\"controls\";a:2:{s:7:\"content\";a:1:{s:31:\"section_post_navigation_content\";a:5:{s:10:\"prev_label\";i:1;s:10:\"next_label\";i:1;s:12:\"show_borders\";i:1;s:5:\"arrow\";i:1;s:10:\"show_arrow\";i:1;}}s:5:\"style\";a:1:{s:11:\"title_style\";a:3:{s:27:\"title_typography_typography\";i:1;s:26:\"title_typography_font_size\";i:1;s:28:\"title_typography_font_weight\";i:1;}}}}}"],"_wp_old_slug":["elementor-30125"],"_elementor_css":["a:6:{s:4:\"time\";i:1775577487;s:5:\"fonts\";a:0:{}s:5:\"icons\";a:1:{i:0;s:0:\"\";}s:20:\"dynamic_elements_ids\";a:1:{i:0;s:7:\"4829f73\";}s:6:\"status\";s:4:\"file\";i:0;s:0:\"\";}"],"_elementor_page_assets":["a:2:{s:6:\"styles\";a:13:{i:0;s:16:\"widget-post-info\";i:1;s:16:\"widget-icon-list\";i:2;s:26:\"elementor-icons-fa-regular\";i:3;s:24:\"elementor-icons-fa-solid\";i:4;s:14:\"widget-heading\";i:5;s:17:\"widget-author-box\";i:6;s:14:\"widget-divider\";i:7;s:21:\"widget-code-highlight\";i:8;s:8:\"e-swiper\";i:9;s:21:\"widget-image-carousel\";i:10;s:12:\"widget-image\";i:11;s:15:\"thegem-bloglist\";i:12;s:22:\"widget-post-navigation\";}s:7:\"scripts\";a:11:{i:0;s:18:\"elementor-frontend\";i:1;s:12:\"prismjs_core\";i:2;s:14:\"prismjs_loader\";i:3;s:17:\"prismjs_normalize\";i:4;s:17:\"highlight_handler\";i:5;s:25:\"prismjs_copy_to_clipboard\";i:6;s:6:\"swiper\";i:7;s:23:\"thegem-items-animations\";i:8;s:11:\"thegem-blog\";i:9;s:19:\"thegem-blog-isotope\";i:10;s:15:\"post-navigation\";}}"],"_elementor_element_cache":["{\"timeout\":1776637767,\"value\":{\"content\":\"\\t\\t<section class=\\\"elementor-section elementor-top-section elementor-element elementor-element-5260c7c elementor-section-boxed elementor-section-height-default elementor-section-height-default\\\" data-id=\\\"5260c7c\\\" data-element_type=\\\"section\\\" data-e-type=\\\"section\\\">\\r\\n\\t\\t\\t\\t\\t\\t<div class=\\\"elementor-container elementor-column-gap-thegem\\\"><div class=\\\"elementor-row\\\">\\r\\n\\t\\t\\t\\t\\t<div class=\\\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-f697649\\\" data-id=\\\"f697649\\\" data-element_type=\\\"column\\\" data-e-type=\\\"column\\\">\\n\\t\\t\\t<div class=\\\"elementor-widget-wrap elementor-element-populated\\\">\\n\\t\\t\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjI1OThhNTAiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJpY29uX2xpc3QiOlt7InNlbGVjdGVkX2ljb24iOnsidmFsdWUiOiJmYXMgZmEtY2FsZW5kYXIiLCJsaWJyYXJ5IjoiZmEtc29saWQifSwiX2lkIjoiOTFhMGY1MiIsImN1c3RvbV9kYXRlX2Zvcm1hdCI6IkYgaiwgWSIsImN1c3RvbV90aW1lX2Zvcm1hdCI6Imc6aSBhIiwibGluayI6IiIsInNob3dfaWNvbiI6Im5vbmUifV0sImljb25fdHlwb2dyYXBoeV90eXBvZ3JhcGh5IjoiY3VzdG9tIiwiX19nbG9iYWxzX18iOnsidGV4dF9jb2xvciI6Imdsb2JhbHNcL2NvbG9ycz9pZD1wcmltYXJ5In19LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJwb3N0LWluZm8ifQ==\\\"][elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjQ4MjlmNzMiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJ0aXRsZSI6IkFkZCBZb3VyIEhlYWRpbmcgVGV4dCBIZXJlIiwiaGVhZGVyX3NpemUiOiJkaXYiLCJ0aGVnZW1faGVhZGluZ19zdHlsZSI6InN0eWxlZC1zdWJ0aXRsZSIsIl9fZHluYW1pY19fIjp7InRpdGxlIjoiW2VsZW1lbnRvci10YWcgaWQ9XCI3YjZhZDE0XCIgbmFtZT1cInRoZWdlbS1wb3N0LWV4Y2VycHRcIiBzZXR0aW5ncz1cIiU3QiU3RFwiXSJ9fSwiZWxlbWVudHMiOltdLCJ3aWRnZXRUeXBlIjoiaGVhZGluZyJ9\\\"][elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjM5MDZhZWEiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJzb3VyY2UiOiJjdXN0b20iLCJhdXRob3JfYXZhdGFyIjp7InVybCI6Imh0dHBzOlwvXC9rbXdsbGNjb20uc3RhZ2Uuc2l0ZVwvd3AtY29udGVudFwvdXBsb2Fkc1wvMjAyNVwvMTBcL0phY29iLmpwZyIsImlkIjozMDIwOCwiYWx0IjoiIiwic291cmNlIjoibGlicmFyeSIsInNpemUiOiIifSwiYXV0aG9yX25hbWUiOiJKYWNvYiBTcXVhdHJpdG8iLCJhdXRob3JfbmFtZV90YWciOiJkaXYiLCJhdXRob3JfYmlvIjoiU2VhcmNoICYgQUkgRW5naW5lZXIgYXQgS01XIFRlY2hub2xvZ3kiLCJsaW5rX3RleHQiOiJBbGwgUG9zdHMiLCJpbWFnZV92ZXJ0aWNhbF9hbGlnbiI6InRvcCIsImltYWdlX2JvcmRlciI6InllcyIsImltYWdlX2JvcmRlcl93aWR0aCI6eyJ1bml0IjoicHgiLCJzaXplIjozLCJzaXplcyI6W119LCJpbWFnZV9ib3JkZXJfcmFkaXVzIjp7InVuaXQiOiJweCIsInNpemUiOjEwMCwic2l6ZXMiOltdfSwibmFtZV9nYXAiOnsidW5pdCI6InB4Iiwic2l6ZSI6MCwic2l6ZXMiOltdfSwiX19nbG9iYWxzX18iOnsiaW1hZ2VfYm9yZGVyX2NvbG9yIjoiZ2xvYmFsc1wvY29sb3JzP2lkPTQyOGYyNzciLCJiaW9fY29sb3IiOiJnbG9iYWxzXC9jb2xvcnM\\\/aWQ9c2Vjb25kYXJ5In0sIl9wYWRkaW5nIjp7InVuaXQiOiJlbSIsInRvcCI6IjIiLCJyaWdodCI6IjIiLCJib3R0b20iOiIyIiwibGVmdCI6IjIiLCJpc0xpbmtlZCI6dHJ1ZX0sIl9wYWRkaW5nX3RhYmxldCI6eyJ1bml0IjoiZW0iLCJ0b3AiOiIiLCJyaWdodCI6IiIsImJvdHRvbSI6IiIsImxlZnQiOiIiLCJpc0xpbmtlZCI6dHJ1ZX0sIl9wYWRkaW5nX21vYmlsZSI6eyJ1bml0IjoiZW0iLCJ0b3AiOiIiLCJyaWdodCI6IiIsImJvdHRvbSI6IiIsImxlZnQiOiIiLCJpc0xpbmtlZCI6dHJ1ZX19LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJhdXRob3ItYm94In0=\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-7a1c601 elementor-widget-divider--view-line flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-divider\\\" data-id=\\\"7a1c601\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"divider.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t<div class=\\\"elementor-divider\\\">\\n\\t\\t\\t<span class=\\\"elementor-divider-separator\\\">\\n\\t\\t\\t\\t\\t\\t<\\\/span>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjU0OTk4ZjQiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+RW50aXRpZXMgYXJlIHJlYWxseSBpbXBvcnRhbnQgZm9yIHNlYXJjaCwgYnV0IHdoYXRcdTIwMTlzIHRoZSBiZXN0IHdheSB0byBkbyBpdD88XC9zcGFuPjxcL3A+PHA+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPlRoZSBhYmlsaXR5IHRvIGFuYWx5emUgYSBwaWVjZSBvZiB0ZXh0IGFuZCBpZGVudGlmeSB0aGUga2V5IGVudGl0aWVzIHdpdGhpbiBpdCBjYW4gaGF2ZSBsb3RzIG9mIHByYWN0aWNhbCBzZWFyY2ggdXNlcyBsaWtlIGltcHJvdmluZyBzZWFyY2ggcmVsZXZhbmN5OyBlbmFibGluZyBmYWNldGluZyBhbmQgZmlsdGVyaW5nIGNsYXNzaWZ5aW5nIGRvY3VtZW50cyBhbmQgZXZlbiBvYmZ1c2NhdGlvbiBvZiBzZW5zaXRpdmUgZGF0YS48XC9zcGFuPjxcL3A+PHA+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPkFzIGEgbmF0dXJhbCBsYW5ndWFnZSBwcm9ibGVtLCBlbnRpdHkgZXh0cmFjdGlvbiBpcyBub3QgbmV3IGJ1dCBpdFx1MjAxOXMgYWx3YXlzIGJlZW4gdHJpY2t5IHRvIGRvIHdlbGwuIFRoZXJlIGFyZSBsb3RzIG9mIHRyYWRpdGlvbmFsIE5MUCBtb2RlbHMgZm9yIGVudGl0eSBleHRyYWN0aW9uIGFuZCByZWNlbnRseSBMTE1zIGhhdmUgc2hvd24gcHJvbWlzaW5nIGFiaWxpdGllcyB0b28uIEZvciBzZWFyY2ggYXBwbGljYXRpb25zIHdlIGFyZSB0eXBpY2FsbHkgdHJ5aW5nIHRvIGJhbGFuY2UgZXhjZWxsZW50IGVudGl0eSBleHRyYWN0aW9uIGFsb25nIHdpdGggb3BlcmF0aW9uYWwgbmVlZHMgbGlrZSBydW5uaW5nIGZhc3QgYW5kIG5vdCBjb25zdW1pbmcgdG9vIG1hbnkgcmVzb3VyY2VzLiBTbyB3aGF0IGlzIHRoZSBiZXN0IHdheSB0byBkbyBlbnRpdHkgZXh0cmFjdGlvbiBmb3IgbW9kZXJuIHNlYXJjaCBhcHBsaWNhdGlvbnM\\\/PFwvc3Bhbj48XC9wPjxwPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj5JbiB0aGlzIGJsb2cgcG9zdCB3ZSBhcmUgZ29pbmcgdG8gY29tcGFyZSB0cmFkaXRpb25hbCBOTFAgbW9kZWxzIHdpdGggTExNcyB0byBzZWUgaG93IHRoZXkgbWVhc3VyZSB1cC4gV2UnbGwgZm9jdXMgb24gZXh0cmFjdGluZyB0aGUgbmFtZXMgb2YgcGVvcGxlLCBvcmdhbml6YXRpb25zLCBvciBsb2NhdGlvbnMgd2l0aGluIGEgYm9keSBvZiB0ZXh0LiBXZVx1MjAxOWxsIGRpc2N1c3Mgd2F5cyBlbnRpdHkgZXh0cmFjdGlvbiBjYW4gaW1wcm92ZSB0aGUgc2VhcmNoIGV4cGVyaWVuY2UsIGFuYWx5emUgdGhlIHBlcmZvcm1hbmNlIG9mIHRyYWRpdGlvbmFsIG1vZGVscyBhbmQgbGFyZ2UgbGFuZ3VhZ2UgbW9kZWxzLCBydW4gYSBmZXcgZXhwZXJpbWVudHMsIGFuZCBjb25jbHVkZSB3aXRoIGEgcmV2aWV3IG9mIG91ciBmaW5kaW5nczxcL3NwYW4+PFwvcD4ifSwiZWxlbWVudHMiOltdLCJ3aWRnZXRUeXBlIjoidGV4dC1lZGl0b3IifQ==\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-8717084 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"8717084\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<h1 class=\\\"elementor-heading-title elementor-size-large\\\">A Simple Example<\\\/h1>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjU4MzE0NjUiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+SW1hZ2luZSB3ZSBoYXZlIGZvdXIgZG9jdW1lbnRzIHdlIHdhbnQgdG8gaW5kZXg6PFwvc3Bhbj48XC9wPiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0ZXh0LWVkaXRvciJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-2e5a0c6 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-code-highlight\\\" data-id=\\\"2e5a0c6\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"code-highlight.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t<div class=\\\"prismjs-default copy-to-clipboard \\\">\\n\\t\\t\\t<pre data-line=\\\"\\\" class=\\\"highlight-height language-json \\\">\\n\\t\\t\\t\\t<code readonly=\\\"true\\\" class=\\\"language-json\\\">\\n\\t\\t\\t\\t\\t<xmp>{\\r\\n  \\u201cid\\u201d: 1\\r\\n  \\u201ctext\\u201d: \\u201cWill, are you going to the store today?\\u201d\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 2\\r\\n  \\u201ctext\\u201d: \\u201cWill you go to the store today?\\u201d\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 3\\r\\n  \\u201ctext\\u201d: \\u201cI hope you will join us.\\u201d\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 4\\r\\n  \\u201ctext\\u201d: \\u201cIs Hope going to be joining us?\\u201d\\r\\n}\\r\\n\\r\\n\\r\\n<\\\/xmp>\\n\\t\\t\\t\\t<\\\/code>\\n\\t\\t\\t<\\\/pre>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6ImE5NzQzZjYiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+TGV0XHUyMDE5cyBzYXkgeW91IHdhbnQgdG8gc2VhcmNoIGZvciBkb2N1bWVudHMgcmVmZXJlbmNpbmcgYSBjZXJ0YWluIHBlcnNvbi4gSW4gc29tZSBjYXNlcywgeW91IG1pZ2h0IGdldCBhd2F5IHdpdGgganVzdCBzZWFyY2hpbmcgZm9yIHRoZWlyIG5hbWUsIGFzLWlzLCBpbiA8XC9zcGFuPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj50ZXh0PFwvc3Bhbj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+LiBCdXQsIHdoZW4geW91IHNlYXJjaCBmb3IgbmFtZXMgbGlrZSBXaWxsIG9yIEhvcGUsIHlvdXIgc2VhcmNoIGVuZ2luZSB3aWxsIGxpa2VseSByZXR1cm4gZG9jdW1lbnRzIHRoYXQgdXNlIHRoZXNlIHdvcmRzIGluIGEgZGlmZmVyZW50IGNvbnRleHQuIEFzIHlvdSBjYW4gc2VlLCBcdTIwMWN3aWxsXHUyMDFkIGlzIGZvdW5kIGluIGRvY3VtZW50cyAxLCAyLCBhbmQgMywgYnV0IG9ubHkgZG9jdW1lbnQgMSBhY3R1YWxseSByZWZlcmVuY2VzIHNvbWVvbmUgbmFtZWQgV2lsbC4gV2UgcnVuIGludG8gYSBzaW1pbGFyIGlzc3VlIHdoZW4gc2VhcmNoaW5nIGZvciBcdTIwMWNob3BlXHUyMDFkIGFzIHdlbGwuXHUwMGEwPFwvc3Bhbj48XC9wPjxwPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj5IYXZpbmcgc29tZSBpcnJlbGV2YW50IGRvY3VtZW50cyByZXR1cm5lZCBpcywgb2YgY291cnNlLCBub3QgaWRlYWwuIEJ1dCBpbiBwcmFjdGljZSwgdGhlIHByb2JsZW0gbWF5IGJlIG1vcmUgdGhhbiBqdXN0IGEgbWlub3IgbnVpc2FuY2UuIFRlcm1zIGxpa2UgV2lsbCBhbmQgSG9wZSBjb3VsZCBiZSB1c2VkIG1vcmUgb2Z0ZW4gYXMgRW5nbGlzaCB3b3JkcyByYXRoZXIgdGhhbiBuYW1lcy4gRG9jdW1lbnRzIHRoYXQgbmF0dXJhbGx5IHVzZSB0aGUgd29yZCBcdTIwMWN3aWxsXHUyMDFkIG11bHRpcGxlIHRpbWVzIG1pZ2h0IHNjb3JlIGhpZ2hlciB0aGFuIGRvY3VtZW50cyBtZW50aW9uaW5nIGEgcGVyc29uIG5hbWVkIFdpbGwuIE1ha2luZyBtYXR0ZXJzIGV2ZW4gd29yc2UsIHRoZSB0ZXh0IG1pZ2h0IHJlZmVyZW5jZSBXaWxsIHdpdGggcHJvbm91bnMgaW5zdGVhZCBvZiBzdGF0aW5nIGhpcyBuYW1lIHJlcGVhdGVkbHksIGZ1cnRoZXIgZGVjcmVhc2luZyB0aGUgZG9jdW1lbnRcdTIwMTlzIHNlYXJjaCBzY29yZS4gPFwvc3Bhbj48XC9wPjxwPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj5Tb1x1MjAyNiBqdXN0IHJ1bm5pbmcgYSBzZWFyY2ggb24gPFwvc3Bhbj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+XHUyMDFjdGV4dFx1MjAxZDxcL3NwYW4+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPiB3b25cdTIwMTl0IGFsd2F5cyBzdWZmaWNlLiBUbyBhdm9pZCBtYW51YWxseSBwcnVuaW5nIHRocm91Z2ggeW91ciBzZWFyY2ggcmVzdWx0cyB0byByZW1vdmUgdGhlIGlycmVsZXZhbnQgZG9jdW1lbnRzLCB5b3VcdTIwMTlsbCB3YW50IHRvIGVucmljaCB5b3VyIGNvbnRlbnQgYmVmb3JlIGluZGV4aW5nLlx1MDBhMDxcL3NwYW4+PFwvcD48cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+V2l0aCBlbnRpdHkgZXh0cmFjdGlvbiwgd2UgY2FuIGVucmljaCBvdXIgZG9jdW1lbnRzIGxpa2Ugc286PFwvc3Bhbj48XC9wPiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0ZXh0LWVkaXRvciJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-3a9e936 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-code-highlight\\\" data-id=\\\"3a9e936\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"code-highlight.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t<div class=\\\"prismjs-default copy-to-clipboard \\\">\\n\\t\\t\\t<pre data-line=\\\"\\\" class=\\\"highlight-height language-json \\\">\\n\\t\\t\\t\\t<code readonly=\\\"true\\\" class=\\\"language-json\\\">\\n\\t\\t\\t\\t\\t<xmp>{\\r\\n  \\u201cid\\u201d: 1\\r\\n  \\u201ctext\\u201d: \\u201cWill, are you going to the store today?\\u201d\\r\\n  \\u201cpeople\\u201d: [\\u201cWill\\u201d]\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 2\\r\\n  \\u201ctext\\u201d: \\u201cWill you go to the store today?\\u201d\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 3\\r\\n  \\u201ctext\\u201d: \\u201cI hope you will join us.\\u201d\\r\\n}\\r\\n\\r\\n{\\r\\n  \\u201cid\\u201d: 4\\r\\n  \\u201ctext\\u201d: \\u201cIs Hope going to be joining us?\\u201d\\r\\n  \\u201cpeople\\u201d: [\\u201cHope\\u201d]\\r\\n}\\r\\n<\\\/xmp>\\n\\t\\t\\t\\t<\\\/code>\\n\\t\\t\\t<\\\/pre>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6ImM5NWEzYWEiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+SWYgd2UgZXh0cmFjdCBlbnRpdGllcyBmcm9tIHRoZSB0ZXh0LCB3ZSBjYW4gYXR0YWNoIGxpc3RzIG9mIG5hbWVzIG1lbnRpb25lZCB0byB0aGUgZG9jdW1lbnQuIE5vdywgaWYgd2Ugd2FudCB0byBzZWFyY2ggZm9yIGRvY3VtZW50cyB0aGF0IHJlZmVyZW5jZSBzb21lYm9keSBuYW1lZCBXaWxsLCB3ZSB3b3VsZCBzZWFyY2ggb24gPFwvc3Bhbj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+XHUyMDFjcGVvcGxlXHUyMDFkPFwvc3Bhbj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+IGluc3RlYWQgb2YgPFwvc3Bhbj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+XHUyMDFjdGV4dFx1MjAxZDxcL3NwYW4+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPi4gVGhpcyBhbGxvd3MgdXMgdG8gZW5zdXJlIGRvY3VtZW50cyBsaWtlIDIgYW5kIDMgZG9uXHUyMDE5dCBoaW5kZXIgb3VyIHNlYXJjaCBwcm9jZXNzIG9yIGNvbnRhbWluYXRlIG91ciByZXN1bHRzLiBUaGlzIGlzIGRlZmluaXRlbHkgYSBzaW1wbGUgdXNlIGNhc2UgLSBkaWZmZXJlbnQgdXNlIGNhc2VzIHdpbGwgY2VydGFpbmx5IGhhdmUgdW5pcXVlIGNoYWxsZW5nZXMgdG8gb3ZlcmNvbWUuPFwvc3Bhbj48XC9wPiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0ZXh0LWVkaXRvciJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-4722b30 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"4722b30\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<h1 class=\\\"elementor-heading-title elementor-size-large\\\">Our Entity Extraction Approach<\\\/h1>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjkzNzAxNjkiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+VGhlcmUgZXhpc3QgbWFueSBzdGF0aXN0aWNhbFwvcnVsZXMtYmFzZWQgbW9kZWxzIGZvciBlbnRpdHkgZXh0cmFjdGlvbiBhbmQgbmFtZWQgZW50aXR5IHJlY29nbml0aW9uLiBGb3Igb3VyIHRlc3RpbmcsIHdlIHVzZWQgQXBhY2hlIE9wZW5OTFBcdTIwMTlzIDxcL3NwYW4+PGEgaHJlZj1cImh0dHBzOlwvXC9vcGVubmxwLnNvdXJjZWZvcmdlLm5ldFwvbW9kZWxzLTEuNVwvXCI+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPnByZXRyYWluZWQgbW9kZWxzPFwvc3Bhbj48XC9hPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj4gYW5kIFN0YW5mb3JkIENvcmVOTFBcdTIwMTlzIGJ1aWx0LWluIG5hbWVkIGVudGl0eSByZWNvZ25pdGlvbiBtb2RlbHMuIEJ1dCB0aGV5IGFyZW5cdTIwMTl0IHBlcmZlY3QsIGFuZCB3aXRoIHRoZSBldmVyIGluY3JlYXNpbmcgcG9wdWxhcml0eSBvZiBsYXJnZSBsYW5ndWFnZSBtb2RlbHMgKExMTXMpLCB3ZSB0aG91Z2h0IGl0IHdvdWxkIGJlIGludGVyZXN0aW5nIHRvIHNlZSBob3cgYW4gTExNIHBlcmZvcm1zIGVudGl0eSBleHRyYWN0aW9uIGNvbXBhcmVkIGFnYWluc3QgdGhlc2UgbGVnYWN5IGFwcHJvYWNoZXMuIDxcL3NwYW4+PFwvcD48cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+V2UgdGhlb3JpemVkIHRoYXQgYW4gTExNIGhhZCB0aGUgcG90ZW50aWFsIHRvIGV4dHJhY3QgY2VydGFpbiBuYW1lcyB0aGF0IG1vcmUgdHJhZGl0aW9uYWwgbW9kZWxzIG1heSBiZSBsaWtlbHkgdG8gbWlzcy4gKEluIHBhcnRpY3VsYXIsIHdlIGZpZ3VyZWQgYW4gTExNIHdvdWxkIGJlIG1vcmUgbGlrZWx5IHRvIGV4dHJhY3QgYSBcdTIwMWNuZXdlclx1MjAxZCBuYW1lIHRoYXQgd2FzIHVuY29tbW9uIHdoZW4gdGhlIHRyYWRpdGlvbmFsIG1vZGVscyB3ZXJlIHRyYWluZWQuKSBCdXQsIHdlIGFsc28gYmVsaWV2ZWQgYW4gTExNIGNvdWxkIHBvdGVudGlhbGx5IGdldCBcdTIwMWNkaXN0cmFjdGVkXHUyMDFkIGFuZCB1bmRlcnBlcmZvcm0gdGhlIHRyYWRpdGlvbmFsIG1vZGVscyBvbiBsb25nZXIgcGllY2VzIG9mIHRleHQuPFwvc3Bhbj48XC9wPjxwPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj5JbiBvcmRlciB0byB0ZXN0IG91ciB0aGVvcmllcywgd2UgcmFuIGZvdXIgZGlmZmVyZW50IGV4cGVyaW1lbnRzOjxcL3NwYW4+PFwvcD48b2w+PGxpIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIiBhcmlhLWxldmVsPVwiMVwiPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj48c3Ryb25nPkJhc2VsaW5lOjxcL3N0cm9uZz4gV2Ugc3RhcnRlZCBieSBldmFsdWF0aW5nIHRoZSBwZXJmb3JtYW5jZSBvZiBPcGVuTkxQLCBDb3JlTkxQLCBhbmQgR29vZ2xlXHUyMDE5cyA8XC9zcGFuPjxhIGhyZWY9XCJodHRwczpcL1wvb2xsYW1hLmNvbVwvbGlicmFyeVwvZ2VtbWEzXCI+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPmdlbW1hMzxcL3NwYW4+PFwvYT48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+LCBhIHBvcHVsYXIgYW5kIGNhcGFibGUgb3Blbi1zb3VyY2UgTExNLiBUaGUgZGVmYXVsdCB2ZXJzaW9uIG9mIHRoZSBtb2RlbCBpcyBhYm91dCAzIEdCIGluIHNpemUgYW5kIGhhcyByb3VnaGx5IDQgYmlsbGlvbiBwYXJhbWV0ZXJzLCBtYWtpbmcgaXQgc3VpdGFibGUgZm9yIHVzZSBvbiBtb2Rlcm4gaGFyZHdhcmUuIEZvciBlYWNoIG1vZGVsLCB3ZSBldmFsdWF0ZWQgaXRzIHByZWNpc2lvbiwgcmVjYWxsLCBhbmQgRjEgc2NvcmUuPFwvc3Bhbj48XC9saT48bGkgc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiIGFyaWEtbGV2ZWw9XCIxXCI+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPjxzdHJvbmc+TnVtYmVyIG9mIFBhcmFtZXRlcnM6PFwvc3Ryb25nPiBXZSBpbnRyb2R1Y2VkIHR3byBuZXcgdmFyaWFudHMgb2YgZ2VtbWEzIHdpdGggYSBkaWZmZXJlbnQgbnVtYmVyIG9mIHBhcmFtZXRlcnMuIFdlIGRpc2N1c3NlZCBob3cgdXNpbmcgbW9yZSBcLyBsZXNzIHBhcmFtZXRlcnMgYXBwZWFyZWQgdG8gYWZmZWN0IHRoZSByZXN1bHRzLjxcL3NwYW4+PFwvbGk+PGxpIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIiBhcmlhLWxldmVsPVwiMVwiPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj48c3Ryb25nPlR3byBQYXNzPFwvc3Ryb25nPjogVXNpbmcgdGhlIHR3byBzdHJvbmdlc3QgbW9kZWxzIC0gQ29yZU5MUCBhbmQgZ2VtbWEzIC0gd2UgaW5zdHJ1Y3RlZCBnZW1tYTMgdG8gb2JzZXJ2ZSBhbmQgZWRpdCB0aGUgb3V0cHV0IG9mIENvcmVOTFAgYXMgaXQgc2F3IGZpdC5cdTAwYTA8XC9zcGFuPjxcL2xpPjxsaSBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCIgYXJpYS1sZXZlbD1cIjFcIj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+PHN0cm9uZz5BbHRlcm5hdGUgTW9kZWw6PFwvc3Ryb25nPiBXZSBldmFsdWF0ZWQgYW5vdGhlciBwb3B1bGFyIG1vZGVsLCA8XC9zcGFuPjxhIGhyZWY9XCJodHRwczpcL1wvb2xsYW1hLmNvbVwvbGlicmFyeVwvZGVlcHNlZWstcjFcIj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+ZGVlcHNlZWstcjE8XC9zcGFuPjxcL2E+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPiwgYWdhaW5zdCBPcGVuTkxQIGFuZCBDb3JlTkxQLCB0byBzZWUgaWYgdGhlcmUgd2VyZSBhbnkgbm90YWJsZSBkaWZmZXJlbmNlcy48XC9zcGFuPjxcL2xpPjxcL29sPjxwPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj5Gb3IgYWxsIGV4cGVyaW1lbnRzLCB3ZSB1c2VkIGEgcHVibGljbHkgYXZhaWxhYmxlIGFuZCBmdWxseSBhbm5vdGF0ZWQgPFwvc3Bhbj48YSBocmVmPVwiaHR0cHM6XC9cL2dpdGh1Yi5jb21cL2p1YW5kLXJcL2VudGl0eS1yZWNvZ25pdGlvbi1kYXRhc2V0c1wvYmxvYlwvbWFzdGVyXC9kYXRhXC93aWtpZ29sZFwvQ09OTEwtZm9ybWF0XC9kYXRhXC93aWtpZ29sZC5jb25sbC50eHRcIj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+d2lraWdvbGQgZGF0YXNldDxcL3NwYW4+PFwvYT48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+LCBhbGxvd2luZyB1cyB0byBldmFsdWF0ZSB0aGUgbW9kZWxzIGFnYWluc3QgYSBzb3VyY2Ugb2YgdHJ1dGguIEFjcm9zcyB0aGUgMTQwKyB3aWtpIGFydGljbGVzLCB0aGVyZSB3ZXJlIHJvdWdobHkgMywwMDAgd29yZHMgYW5ub3RhdGVkIGFzIGEgcGVyc29uLCBvcmdhbml6YXRpb24sIG9yIGxvY2F0aW9uLlx1MDBhMDxcL3NwYW4+PFwvcD48cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+QXMgcGFydCBvZiBvdXIgZXZhbHVhdGlvbiwgd2Ugd29ya2VkIHdpdGggPFwvc3Bhbj48YSBocmVmPVwiaHR0cHM6XC9cL2dpdGh1Yi5jb21cL2ttd3RlY2hub2xvZ3lcL2x1Y2lsbGVcIj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+THVjaWxsZTxcL3NwYW4+PFwvYT48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+LCBvdXIgb3Blbi1zb3VyY2UgU2VhcmNoIEVUTCBzb2x1dGlvbiB0aGF0IGFsbG93ZWQgdXMgdG8gcGFzcyB0ZXh0IHRocm91Z2ggZW50aXR5IGV4dHJhY3Rpb24gcHJvY2Vzc2VzIGFuZCB1cGRhdGUgZWFjaCBkb2N1bWVudCB3aXRoIHRoZSBvdXRwdXQuIFdlIGNyZWF0ZWQgYSBjdXN0b20gQ29ubmVjdG9yIHRvIHByb2Nlc3MgdGhlIHdpa2lnb2xkIGRhdGFzZXQgaW50byBMdWNpbGxlLCBpbmNsdWRpbmcgdGhlIGFydGljbGVcdTIwMTlzIHRleHQgYXMgd2VsbCBhcyBsaXN0cyBvZiB0aGUgYW5ub3RhdGVkIChvciBcdTIwMWNnb2xkXHUyMDFkKSBwZW9wbGUsIG9yZ2FuaXphdGlvbiwgYW5kIGxvY2F0aW9uIG5hbWVzLlx1MDBhMDxcL3NwYW4+PFwvcD48cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+VG8gcGVyZm9ybSBlbnRpdHkgZXh0cmFjdGlvbiwgd2UgdXNlZCB0aHJlZSBkaWZmZXJlbnQgTHVjaWxsZSBTdGFnZXMuIFdlIGNyZWF0ZWQgYSBjdXN0b20gU3RhZ2UgdG8gZXh0cmFjdCBwZW9wbGUsIG9yZ2FuaXphdGlvbiwgYW5kIGxvY2F0aW9uIG5hbWVzIHVzaW5nIE9wZW5OTFAuIFdlIGRpZCB0aGUgc2FtZSBmb3IgQ29yZU5MUCBhcyB3ZWxsLiBUbyB3b3JrIHdpdGggYW4gTExNLCB3ZSB1c2VkIEx1Y2lsbGVcdTIwMTlzIFByb21wdE9sbGFtYSBTdGFnZSwgd2hpY2ggYWxsb3dzIHlvdSB0byBwcm92aWRlIHBhcnRzIChvciBhbGwpIG9mIGEgZG9jdW1lbnQgdG8gYSBjb21wYXRpYmxlIExMTSBmb3IgZ2VuZXJpYyBlbnJpY2htZW50LiBUaGUgbW9kZWwgd2FzIGluc3RydWN0ZWQgdG8gcmVhZCB0aGUgc291cmNlIHRleHQgYW5kIG91dHB1dCBhIEpTT04gb2JqZWN0IGluY2x1ZGluZyB0aGUgbmFtZXMgb2YgcGVvcGxlLCBvcmdhbml6YXRpb25zLCBhbmQgbG9jYXRpb25zIG1lbnRpb25lZCBpbiB0aGUgZG9jdW1lbnQuIEx1Y2lsbGUgdGhlbiBpbnRlZ3JhdGVkIHRoZSBtb2RlbFx1MjAxOXMgSlNPTiByZXNwb25zZSBpbnRvIHRoZSBkb2N1bWVudC4gVGhlIG1vZGVscyBkaWQgbm90IGhhdmUgYWNjZXNzIHRvIHRoZSBvdXRwdXQgb2Ygb3RoZXIgU3RhZ2VzIC0gdGhleSBvbmx5IHNhdyB0aGUgc291cmNlIHRleHQuXHUwMGEwPFwvc3Bhbj48XC9wPjxwPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj5IZXJlXHUyMDE5cyBhbiBleGFtcGxlIG9mIHdoYXQgYSBmaW5hbGl6ZWQgZG9jdW1lbnQgbG9va2VkIGxpa2UsIGFmdGVyIHdlIG5vcm1hbGl6ZWQgdGhlIG91dHB1dCBmb3IgZXZhbHVhdGlvbjo8XC9zcGFuPjxcL3A+In0sImVsZW1lbnRzIjpbXSwid2lkZ2V0VHlwZSI6InRleHQtZWRpdG9yIn0=\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-bc53fdf flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-code-highlight\\\" data-id=\\\"bc53fdf\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"code-highlight.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t<div class=\\\"prismjs-default copy-to-clipboard \\\">\\n\\t\\t\\t<pre data-line=\\\"\\\" class=\\\"highlight-height language-javascript \\\">\\n\\t\\t\\t\\t<code readonly=\\\"true\\\" class=\\\"language-javascript\\\">\\n\\t\\t\\t\\t\\t<xmp>{\\r\\n  \\u201ctext\\u201d: \\u201c010 is the tenth album from Japanese Punk Techno band The Mad Capsule Markets . This album proved to be more commercial and more techno-based than Osc-Dis , with heavily synthesized songs like Introduction 010 and Come . Founding member Kojima Minoru played guitar on Good Day , and Wardanceis cover of a song by UK post punk industrial band Killing Joke . XXX can of This had a different meaning , and most people did n't understand what the song was about . it was later explained that the song was about Cannabis ( ' can of this ' sounding like Cannabis when said faster ) it is uncertain if they were told to change the lyric like they did on P.O.P and HUMANITY . UK Edition came with the OSC-DIS video , and most of the tracks were re-engineered .\\u201d\\r\\n  \\u201copenNLP_people\\u201d: []\\r\\n  \\u201ccoreNLP_people\\u201d: [\\u201ckojima\\u201d, \\u201cminoru\\u201d]\\r\\n  \\u201collama_people\\u201d: []\\r\\n  \\u201cgold_people\\u201d: [\\u201ckojima\\u201d, \\u201cminoru\\u201d]\\r\\n  \\u201copenNLP_organizations\\u201d: [\\u201cuk\\u201d, \\u201ckilling\\u201d, \\u201cjoke\\u201d, \\u201cfounding\\u201d]\\r\\n  \\u201ccoreNLP_organizations\\u201d: []\\r\\n  \\u201collama_organizations\\u201d: [\\u201cthe\\u201d, \\u201cmad\\u201d, \\u201ccapsule\\u201d, \\u201cmarkets\\u201d, \\u201cmeta\\u201d, \\u201ckilling\\u201d, \\u201cjoke\\u201d]\\r\\n  \\u201cgold_organizations\\u201d: [\\u201cthe\\u201d, \\u201ckilling\\u201d, \\u201cmad\\u201d, \\u201cmarkets\\u201d, \\u201ccapsule\\u201d, \\u201cjoke\\u201d]\\r\\n  \\u201copenNLP_locations\\u201d: []\\r\\n  \\u201ccoreNLP_locations\\u201d: [\\u201cuk\\u201d]\\r\\n  \\u201collama_locations\\u201d: [\\u201cuk\\u201d]\\r\\n  \\u201cgold_locations\\u201d: [\\u201cuk\\u201d]\\r\\n}\\r\\n<\\\/xmp>\\n\\t\\t\\t\\t<\\\/code>\\n\\t\\t\\t<\\\/pre>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6ImI1ZTMyMTEiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+Rm9yIHRoZSBwdXJwb3NlcyBvZiBldmFsdWF0aW9uLCB3ZSBkaWQgbm90IGluZGV4IHRoZSBkb2N1bWVudHMgaW50byBhIHNlYXJjaCBlbmdpbmUuIEluc3RlYWQsIHRoZXkgd2VyZSBpbmRleGVkIGludG8gYSBDU1YsIHdoaWNoIHN0b3JlZCB0aGUgb3JpZ2luYWwgdGV4dCwgYW5ub3RhdGVkIGVudGl0aWVzLCBhbmQgb3V0cHV0IGZyb20gZWFjaCBtb2RlbC4gV2UgdGhlbiByYW4gYSBjdXN0b20gc2NyaXB0IHRvIGFuYWx5emUgdGhlIG1vZGVsc1x1MjAxOSBwZXJmb3JtYW5jZS4gVXNpbmcgdGhlIGFubm90YXRlZCBwZW9wbGUsIG9yZ2FuaXphdGlvbnMsIGFuZCBsb2NhdGlvbnMgZnJvbSB0aGUgd2lraWdvbGQgZGF0YXNldCwgd2Ugd2VyZSBhYmxlIHRvIGNvbXB1dGUgc29tZSBrZXkgbWV0cmljcyBmb3IgZWFjaCBtb2RlbDo8XC9zcGFuPjxcL3A+PHVsPjxsaSBhcmlhLWxldmVsPVwiMVwiPjxiPlByZWNpc2lvbjxcL2I+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPiAtIFdoYXQgcGVyY2VudGFnZSBvZiB0aGUgcGVyc29uXC9vcmdhbml6YXRpb25cL2xvY2F0aW9uIG5hbWVzIG91dHB1dCBieSBhIG1vZGVsIHdlcmUgYW5ub3RhdGVkIGFzIHN1Y2ggaW4gdGhlIGRhdGFzZXQ\\\/PFwvc3Bhbj48XC9saT48XC91bD48dWw+PGxpIGFyaWEtbGV2ZWw9XCIxXCI+PGI+UmVjYWxsPFwvYj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+IC0gV2hhdCBwZXJjZW50YWdlIG9mIHRoZSBhbm5vdGF0ZWQgcGVyc29uXC9vcmdhbml6YXRpb25cL2xvY2F0aW9uIG5hbWVzIGluIHRoZSBkYXRhc2V0IHdlcmUgb3V0cHV0IGJ5IHRoZSBtb2RlbD88XC9zcGFuPjxcL2xpPjxcL3VsPjx1bD48bGkgc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiIGFyaWEtbGV2ZWw9XCIxXCI+PGI+RjEgPFwvYj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+LSBUaGUgXHUyMDFjaGFybW9uaWMgbWVhblx1MjAxZCBvZiBwcmVjaXNpb24gYW5kIHJlY2FsbC4gQ29uc2lkZXJlZCBhIHNvbGlkIG92ZXJhbGwgaW5kaWNhdG9yIG9mIGEgbW9kZWxcdTIwMTlzIHBlcmZvcm1hbmNlLlx1MDBhMDxiciBcLz48XC9zcGFuPjxcL2xpPjxsaT48Yj5VbmlxdWUgR29sZCBXb3JkczxcL2I+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPiAtIEhvdyBtYW55IGdvbGQgbmFtZXMgZGlkIGEgbW9kZWwgbWVudGlvbiB0aGF0IG5vIG90aGVyIG1vZGVsIGRpZD88XC9zcGFuPjxcL2xpPjxcL3VsPiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0ZXh0LWVkaXRvciJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-b827d24 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"b827d24\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<h1 class=\\\"elementor-heading-title elementor-size-large\\\">ExperIminets &amp; Results<\\\/h1>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<div class=\\\"elementor-element elementor-element-719512e flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"719512e\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<h2 class=\\\"elementor-heading-title elementor-size-medium\\\">Experiment 1: Number of Parameters<\\\/h2>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6Ijg4NjFhMmEiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+V2UgYmVnYW4gYnkgY3JlYXRpbmcgYSBwaXBlbGluZSB0aGF0IHVzZWQgdGhyZWUgZGlmZmVyZW50IG1vZGVsczogT3Blbk5MUCwgQ29yZU5MUCwgYW5kIGdlbW1hMy4gRWFjaCBtb2RlbCByYW4gaW5kZXBlbmRlbnRseSBvZiB0aGUgb3RoZXIsIG1lYW5pbmcgdGhleSB3ZXJlIG5vdCBhd2FyZSBvZiBlYWNoIG90aGVyXHUyMDE5cyBvdXRwdXQuIEhlcmUgYXJlIHRoZSBwcmVjaXNpb24sIHJlY2FsbCwgYW5kIEYxIHNjb3JlcyBmb3IgZWFjaCBtb2RlbDo8XC9zcGFuPjxcL3A+In0sImVsZW1lbnRzIjpbXSwid2lkZ2V0VHlwZSI6InRleHQtZWRpdG9yIn0=\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-a33b483 elementor-arrows-position-inside elementor-pagination-position-outside flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image-carousel\\\" data-id=\\\"a33b483\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-settings=\\\"{&quot;slides_to_show&quot;:&quot;1&quot;,&quot;navigation&quot;:&quot;both&quot;,&quot;autoplay&quot;:&quot;yes&quot;,&quot;pause_on_hover&quot;:&quot;yes&quot;,&quot;pause_on_interaction&quot;:&quot;yes&quot;,&quot;autoplay_speed&quot;:5000,&quot;infinite&quot;:&quot;yes&quot;,&quot;effect&quot;:&quot;slide&quot;,&quot;speed&quot;:500}\\\" data-widget_type=\\\"image-carousel.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t<div class=\\\"elementor-image-carousel-wrapper swiper\\\" role=\\\"region\\\" aria-roledescription=\\\"carousel\\\" aria-label=\\\"Image Carousel\\\" dir=\\\"ltr\\\">\\n\\t\\t\\t<div class=\\\"elementor-image-carousel swiper-wrapper\\\" aria-live=\\\"off\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t<div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"1 of 3\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1gemma3.png\\\" alt=\\\"exp1gemma3\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"2 of 3\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1opennlp.png\\\" alt=\\\"exp1opennlp\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"3 of 3\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1corenlp.png\\\" alt=\\\"exp1corenlp\\\" \\\/><\\\/figure><\\\/div>\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<div class=\\\"elementor-swiper-button elementor-swiper-button-prev\\\" role=\\\"button\\\" tabindex=\\\"0\\\">\\n\\t\\t\\t\\t\\t\\t<i aria-hidden=\\\"true\\\" class=\\\"eicon-chevron-left\\\"><\\\/i>\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t<div class=\\\"elementor-swiper-button elementor-swiper-button-next\\\" role=\\\"button\\\" tabindex=\\\"0\\\">\\n\\t\\t\\t\\t\\t\\t<i aria-hidden=\\\"true\\\" class=\\\"eicon-chevron-right\\\"><\\\/i>\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t<div class=\\\"swiper-pagination\\\"><\\\/div>\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjBkMGViYjQiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+Q2xlYXJseSwgQ29yZU5MUCBpcyB0aGUgc3Ryb25nZXN0IGNvbnRlbmRlciBoZXJlLCB3aXRoIGdlbW1hMyBpbiBhIGNsb3NlIHNlY29uZC4gQm90aCBoYWQgc2ltaWxhciBGMSBzY29yZXMgb2YgYWJvdXQgfjAuNzUuIE9wZW5OTFAgd2Fzblx1MjAxOXQgdGhlIHN0cm9uZ2VzdCBjb250ZW5kZXIgd2l0aCBhIGxvd2VyIEYxIHNjb3JlLiBXZSBzaG91bGQgYWxzbyBjb25zaWRlciB0aGUgbGF0ZW5jeSBhc3NvY2lhdGVkIHdpdGggcnVubmluZyB0aGUgTExNLjxcL3NwYW4+PFwvcD48cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+T24gYXZlcmFnZSwgZ2VtbWEzIHRvb2sgYWJvdXQgOCBzZWNvbmRzIHRvIHJlc3BvbmQgcGVyIERvY3VtZW50LCBzbG93aW5nIHRoZSBwaXBlbGluZSBkb3duIHN1YnN0YW50aWFsbHkuIChUaGUgZXhwZXJpbWVudCB3YXMgcnVuIG9uIGFuIEFwcGxlIE0xIFBybyB3aXRoIDE2IEdCIG9mIFJBTS4pPFwvc3Bhbj48XC9wPjxwPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj5XZSBhbHNvIGFuYWx5emVkIHRoZSBlZmZlY3Qgb2YgdGV4dCBsZW5ndGggb24gbW9kZWwgcGVyZm9ybWFuY2UuIEhlcmUsIHdlIGNvbnNpZGVyZWQganVzdCB0aGUgdG9wIHBlcmZvcm1pbmcgbW9kZWxzIC0gQ29yZU5MUCBhbmQgZ2VtbWEzLiBXZSBjYWxjdWxhdGVkIHRoZSBzYW1lIG1ldHJpY3MgKHByZWNpc2lvbiwgcmVjYWxsLCBGMSkgPFwvc3Bhbj48aT48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+Zm9yIGVhY2g8XC9zcGFuPjxcL2k+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPiB3aWtpIGFydGljbGUuIFRoZXJlIHdlcmUgYSBmZXcgYXJ0aWNsZXMgd2l0aCA5MDArIHdvcmRzIHRoYXQgd2UgZXhjbHVkZWQgdG8gYXZvaWQgc2tld2luZyB0aGUgcmVzdWx0cy4gV2UgYWxzbyBleGNsdWRlZCBhcnRpY2xlcyB3aXRoIGxlc3MgdGhhbiAxMDAgd29yZHMuIFNpbmNlIHRoZXNlIGRvY3VtZW50cyB3ZXJlIHZlcnkgc2hvcnQsIHRoZXkgdXN1YWxseSBqdXN0IGRpZG5cdTIwMTl0IHJlZmVyZW5jZSBuYW1lcyBvZiBhIGNlcnRhaW4gdHlwZS4gQXMgYSByZXN1bHQsIHRoZSBtb2RlbCBzY29yZXMgd2VyZSBwcmltYXJpbHkgZWl0aGVyIHplcm8gb3Igb25lLCB3aGljaCBtYWRlIHRoZSByZXN1bHRzIHZlcnkgdm9sYXRpbGUgYW5kIGRpZmZpY3VsdCB0byBvYnNlcnZlOjxcL3NwYW4+PFwvcD4ifSwiZWxlbWVudHMiOltdLCJ3aWRnZXRUeXBlIjoidGV4dC1lZGl0b3IifQ==\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-461ca00 elementor-arrows-position-inside elementor-pagination-position-outside flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image-carousel\\\" data-id=\\\"461ca00\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-settings=\\\"{&quot;slides_to_show&quot;:&quot;1&quot;,&quot;navigation&quot;:&quot;both&quot;,&quot;autoplay&quot;:&quot;yes&quot;,&quot;pause_on_hover&quot;:&quot;yes&quot;,&quot;pause_on_interaction&quot;:&quot;yes&quot;,&quot;autoplay_speed&quot;:5000,&quot;infinite&quot;:&quot;yes&quot;,&quot;effect&quot;:&quot;slide&quot;,&quot;speed&quot;:500}\\\" data-widget_type=\\\"image-carousel.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t<div class=\\\"elementor-image-carousel-wrapper swiper\\\" role=\\\"region\\\" aria-roledescription=\\\"carousel\\\" aria-label=\\\"Image Carousel\\\" dir=\\\"ltr\\\">\\n\\t\\t\\t<div class=\\\"elementor-image-carousel swiper-wrapper\\\" aria-live=\\\"off\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t<div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"1 of 6\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_gemma3_f1.png\\\" alt=\\\"exp1textlength_gemma3_f1\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"2 of 6\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_corenlp_f1.png\\\" alt=\\\"exp1textlength_corenlp_f1\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"3 of 6\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_gemma3_recall.png\\\" alt=\\\"exp1textlength_gemma3_recall\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"4 of 6\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_corenlp_recall.png\\\" alt=\\\"exp1textlength_corenlp_recall\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"5 of 6\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_gemma3_precision.png\\\" alt=\\\"exp1textlength_gemma3_precision\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"6 of 6\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp1textlength_corenlp_precision.png\\\" alt=\\\"exp1textlength_corenlp_precision\\\" \\\/><\\\/figure><\\\/div>\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<div class=\\\"elementor-swiper-button elementor-swiper-button-prev\\\" role=\\\"button\\\" tabindex=\\\"0\\\">\\n\\t\\t\\t\\t\\t\\t<i aria-hidden=\\\"true\\\" class=\\\"eicon-chevron-left\\\"><\\\/i>\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t<div class=\\\"elementor-swiper-button elementor-swiper-button-next\\\" role=\\\"button\\\" tabindex=\\\"0\\\">\\n\\t\\t\\t\\t\\t\\t<i aria-hidden=\\\"true\\\" class=\\\"eicon-chevron-right\\\"><\\\/i>\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t<div class=\\\"swiper-pagination\\\"><\\\/div>\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjRkOGVjNDkiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+QXMgZXhwZWN0ZWQsIHRoZSBkYXRhIGhlcmUgd2FzIGEgYml0IHNjYXR0ZXJlZC4gU2luY2UgbWFueSBwb2ludHMgb3ZlcmxhcHBlZCBhdCB0aGUgdG9wIGFuZCBib3R0b20gb2YgZWFjaCBjaGFydCwgd2UgZGlkIGluY2x1ZGUgbGluZXMgb2YgYmVzdCBmaXQgdG8gaGVscCB2aXN1YWxpemUgdGhlIG92ZXJhbGwgdHJlbmQuIEhvd2V2ZXIsIHRoZXNlIGxpbmVzIHNob3VsZCBiZSBpbnRlcnByZXRlZCBjYXV0aW91c2x5LCBhcyB0aGV5IGFsbCBoYWQgYSA8XC9zcGFuPjxpPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj52ZXJ5PFwvc3Bhbj48XC9pPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj4gbG93IFItc3F1YXJlZCB2YWx1ZS4gSW4gb3RoZXIgd29yZHMsIHRoZSBsZW5ndGggb2YgYSBwaWVjZSBvZiB0ZXh0IHNob3VsZG5cdTIwMTl0IGJlIHVzZWQgdG8gc2luZ3VsYXJseSBwcmVkaWN0IHRoZSByZWNhbGwsIHByZWNpc2lvbiwgb3IgRjEgc2NvcmUgeW91XHUyMDE5bGwgZ2V0IGZyb20gYSBtb2RlbC48XC9zcGFuPjxcL3A+PHA+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPkFnYWluLCB0aGVzZSBjaGFydHMgc2hvdWxkIGJlIG9ic2VydmVkIHdpdGggY2F1dGlvbiwgYXMgdGhlcmUgd2VyZSBhIG11bHRpdHVkZSBvZiBmYWN0b3JzIGF0IHBsYXkgaGVyZS4gQnV0LCBpdCBkb2VzIHNlZW0gcmVhc29uYWJsZSB0byBzdWdnZXN0IHRoYXQgdGhlcmUgd2FzIHNvbWUgc29ydCBvZiByZWxhdGlvbnNoaXAgYmV0d2VlbiBsb25nZXIgdGV4dCBhbmQgTExNIHVuZGVycGVyZm9ybWFuY2UuIENvcmVOTFAsIG9uIHRoZSBvdGhlciBoYW5kLCBhcHBlYXJzIHRvIGhhdmUgYmVlbiByZW1hcmthYmx5IGNvbnNpc3RlbnQuPFwvc3Bhbj48XC9wPiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0ZXh0LWVkaXRvciJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-b244a1e flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"b244a1e\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<h2 class=\\\"elementor-heading-title elementor-size-medium\\\">Experiment 2: Number of Parameters<\\\/h2>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjNlZThmYzYiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+T3VyIG5leHQgcGlwZWxpbmUgaW5jbHVkZWQgZml2ZSBkaWZmZXJlbnQgbW9kZWxzIChPcGVuTkxQLCBDb3JlTkxQLCBhbmQgdGhyZWUgdmFyaWFudHMgb2YgZ2VtbWEzKS4gSW4gYWRkaXRpb24gdG8gdGhlIHByZXZpb3VzIHBpcGVsaW5lXHUyMDE5cyBtb2RlbHMsIHdlIGFkZGVkIGEgc21hbGxlciB2YXJpYW50IG9mIGdlbW1hMywgPFwvc3Bhbj48YSBocmVmPVwiaHR0cHM6XC9cL29sbGFtYS5jb21cL2xpYnJhcnlcL2dlbW1hMzoxYlwiPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj5nZW1tYTM6MWI8XC9zcGFuPjxcL2E+PHNwYW4gc3R5bGU9XCJmb250LXdlaWdodDogNDAwO1wiPiwgYW5kIGEgbGFyZ2VyIHZhcmlhbnQgb2YgZ2VtbWEzLCA8XC9zcGFuPjxhIGhyZWY9XCJodHRwczpcL1wvb2xsYW1hLmNvbVwvbGlicmFyeVwvZ2VtbWEzOjEyYlwiPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj5nZW1tYTM6MTJiPFwvc3Bhbj48XC9hPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj4uIEFzIHdlIG1lbnRpb25lZCBhYm92ZSwgPFwvc3Bhbj48YSBocmVmPVwiaHR0cHM6XC9cL29sbGFtYS5jb21cL2xpYnJhcnlcL2dlbW1hMzpsYXRlc3RcIj48c3BhbiBzdHlsZT1cImZvbnQtd2VpZ2h0OiA0MDA7XCI+Z2VtbWEzPFwvc3Bhbj48XC9hPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj4sIGhhZCA0IGJpbGxpb24gcGFyYW1ldGVycyBhbmQgd2FzIGEgbGl0dGxlIG1vcmUgdGhhbiAzIEdCIGluIHNpemUuIFRoZSBzbWFsbGVyIHZhcmlhbnQsIGdlbW1hMzoxYiwgaGFkIDEgYmlsbGlvbiBwYXJhbWV0ZXJzIGFuZCB3YXMgbGVzcyB0aGFuIDEgR0IgaW4gc2l6ZS4gVGhlIGxhcmdlciB2YXJpYW50LCA8XC9zcGFuPjxhIGhyZWY9XCJodHRwczpcL1wvb2xsYW1hLmNvbVwvbGlicmFyeVwvZ2VtbWEzOjEyYlwiPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj5nZW1tYTM6MTJiPFwvc3Bhbj48XC9hPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj4sIGhhZCAxMiBiaWxsaW9uIHBhcmFtZXRlcnMgYW5kIHdhcyBhYm91dCA4IEdCIGluIHNpemUuPFwvc3Bhbj48XC9wPjxwPjxzcGFuIHN0eWxlPVwiZm9udC13ZWlnaHQ6IDQwMDtcIj4gSGVyZSBhcmUgdGhlIHJlc3VsdHMgd2UgZm91bmQ6PFwvc3Bhbj48XC9wPiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0ZXh0LWVkaXRvciJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-23914b5 elementor-arrows-position-inside elementor-pagination-position-outside flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image-carousel\\\" data-id=\\\"23914b5\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-settings=\\\"{&quot;slides_to_show&quot;:&quot;1&quot;,&quot;navigation&quot;:&quot;both&quot;,&quot;autoplay&quot;:&quot;yes&quot;,&quot;pause_on_hover&quot;:&quot;yes&quot;,&quot;pause_on_interaction&quot;:&quot;yes&quot;,&quot;autoplay_speed&quot;:5000,&quot;infinite&quot;:&quot;yes&quot;,&quot;effect&quot;:&quot;slide&quot;,&quot;speed&quot;:500}\\\" data-widget_type=\\\"image-carousel.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t<div class=\\\"elementor-image-carousel-wrapper swiper\\\" role=\\\"region\\\" aria-roledescription=\\\"carousel\\\" aria-label=\\\"Image Carousel\\\" dir=\\\"ltr\\\">\\n\\t\\t\\t<div class=\\\"elementor-image-carousel swiper-wrapper\\\" aria-live=\\\"off\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t<div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"1 of 5\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_gemma3_12b-1.png\\\" alt=\\\"exp2_gemma3_12b\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"2 of 5\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_gemma3-1.png\\\" alt=\\\"exp2_gemma3\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"3 of 5\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_gemma3_1b-1.png\\\" alt=\\\"exp2_gemma3_1b\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"4 of 5\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_opennlp.png\\\" alt=\\\"exp2_opennlp\\\" \\\/><\\\/figure><\\\/div><div class=\\\"swiper-slide\\\" role=\\\"group\\\" aria-roledescription=\\\"slide\\\" aria-label=\\\"5 of 5\\\"><figure class=\\\"swiper-slide-inner\\\"><img class=\\\"swiper-slide-image\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_corenlp.png\\\" alt=\\\"exp2_corenlp\\\" \\\/><\\\/figure><\\\/div>\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<div class=\\\"elementor-swiper-button elementor-swiper-button-prev\\\" role=\\\"button\\\" tabindex=\\\"0\\\">\\n\\t\\t\\t\\t\\t\\t<i aria-hidden=\\\"true\\\" class=\\\"eicon-chevron-left\\\"><\\\/i>\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t<div class=\\\"elementor-swiper-button elementor-swiper-button-next\\\" role=\\\"button\\\" tabindex=\\\"0\\\">\\n\\t\\t\\t\\t\\t\\t<i aria-hidden=\\\"true\\\" class=\\\"eicon-chevron-right\\\"><\\\/i>\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t<div class=\\\"swiper-pagination\\\"><\\\/div>\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjQ2YjhjOTAiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD5BZ2FpbiwgZ2VtbWEzIGFuZCBDb3JlTkxQIGFyZSB0aGUgc3Ryb25nZXN0IGNvbnRlbmRlcnMgaGVyZS4gVGhlIHNtYWxsZXN0IExMTSwgZ2VtbWEzOjFiLCBkaWRuXHUyMDE5dCBkbyB3ZWxsIC0gbm90IG9ubHkgZGlkIGl0IGhhdmUgYSBwb29yIEYxIHNjb3JlLCBidXQgd2UgZm91bmQgaXQgd2FzIGFjdHVhbGx5IHN0cnVnZ2xpbmcgdG8gZm9sbG93IG91ciBpbnN0cnVjdGlvbnMuIFN1cnByaXNpbmdseSwgdGhlIGxhcmdlc3QgTExNLCBnZW1tYTM6MTJiLCB3YXMgYWN0dWFsbHkgYSBiaXQgd29yc2UgdGhhbiB0aGUgbWVkaXVtIHZhcmlhbnQsIGdlbW1hMy4gQ29tcGFyZWQgdG8gZ2VtbWEzLCBnZW1tYTM6MTJiIGhhZCBhIHNvbWV3aGF0IGhpZ2hlciBwcmVjaXNpb24gYnV0IGEgbm90YWJseSBsb3dlciByZWNhbGwuIEl0IHNlZW1zIHRoYXQgdGhpcyBsYXJnZXIgbW9kZWwgd2FzIGEgYml0IHRvbyBjYXV0aW91cyB3aGVuIGVuZ2FnaW5nIHdpdGggdGhlIHNvdXJjZSB0ZXh0LjxcL3A+PHAgaWQ9XCJlbWJlcjM3NVwiIGNsYXNzPVwiZW1iZXItdmlldyByZWFkZXItdGV4dC1ibG9ja19fcGFyYWdyYXBoXCI+Rm9yIHRoaXMgZXhwZXJpbWVudCwgd2UgYWxzbyBjYWxjdWxhdGVkIHRoZSBudW1iZXIgb2YgXHUyMDFjZ29sZFx1MjAxZCB3b3JkcyB0aGF0IHdlcmUgdW5pcXVlbHkgbWVudGlvbmVkIGJ5IGVhY2ggbW9kZWwuIDxcL3A+In0sImVsZW1lbnRzIjpbXSwid2lkZ2V0VHlwZSI6InRleHQtZWRpdG9yIn0=\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-01e24bd flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\\\" data-id=\\\"01e24bd\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"image.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<img width=\\\"793\\\" height=\\\"427\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_goldwords.png\\\" class=\\\"attachment-large size-large wp-image-30235\\\" alt=\\\"\\\" srcset=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_goldwords.png 793w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_goldwords-300x162.png 300w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp2_goldwords-768x414.png 768w\\\" sizes=\\\"(max-width: 793px) 100vw, 793px\\\" \\\/>\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjgxY2U1YmIiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD5BcyBleHBlY3RlZCwgQ29yZU5MUCBhbmQgZ2VtbWEzIHBpY2sgdXAgb24gdGhlIG1vc3QgdW5pcXVlIGdvbGQgd29yZHMuIEludGVyZXN0aW5nbHksIHRoZSBzbWFsbCBhbmQgbGFyZ2UgZ2VtbWEgdmFyaWFudHMgaGFkIHRoZSBmZXdlc3QgdW5pcXVlIGdvbGQgd29yZHMgLSBldmVuIGxlc3MgdGhhbiBPcGVuTkxQLjxcL3A+In0sImVsZW1lbnRzIjpbXSwid2lkZ2V0VHlwZSI6InRleHQtZWRpdG9yIn0=\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-85e041a flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"85e041a\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<h2 class=\\\"elementor-heading-title elementor-size-medium\\\">Experiment 3: Two Pass <\\\/h2>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjRmZGMzYTQiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD5JbiB0aGUgcHJldmlvdXMgZXhwZXJpbWVudCwgd2Ugc2F3IHRoYXQgZ2VtbWEzIGxpc3RlZCB+ODUgXHUyMDFjZ29sZFx1MjAxZCB3b3JkcyB0aGF0IG5vIG90aGVyIG1vZGVsIGRpZC4gQ29yZU5MUCB1bmlxdWVseSBsaXN0ZWQgfjExNSBcdTIwMWNnb2xkXHUyMDFkIHdvcmRzLiBXZSB3b25kZXJlZCBpZiBhIGJldHRlciBvdmVyYWxsIHJlc3VsdCBjb3VsZCBiZSBhY2hpZXZlZCBieSBoYXZpbmcgdGhlIHR3byBtb2RlbHMgYWN0dWFsbHkgd29yayB0b2dldGhlciB0byBpbXByb3ZlIHRoZWlyIG91dHB1dC4gSWRlYWxseSwgYW4gTExNIGNvdWxkIGNhdGNoIHNvbWUgb2YgdGhlc2UgXHUyMDFjYWRkaXRpb25hbFx1MjAxZCBuYW1lcyAoaW5jcmVhc2luZyByZWNhbGwpIGFuZCBtYWtlIHNvbWUgbWlub3IgY2hhbmdlcyB0byBDb3JlTkxQXHUyMDE5cyBvdXRwdXQgKGluY3JlYXNpbmcgcHJlY2lzaW9uKS48XC9wPiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0ZXh0LWVkaXRvciJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-8fd13b7 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-code-highlight\\\" data-id=\\\"8fd13b7\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"code-highlight.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t<div class=\\\"prismjs-default copy-to-clipboard \\\">\\n\\t\\t\\t<pre data-line=\\\"\\\" class=\\\"highlight-height language-javascript \\\">\\n\\t\\t\\t\\t<code readonly=\\\"true\\\" class=\\\"language-javascript\\\">\\n\\t\\t\\t\\t\\t<xmp>(LLM Request)\\r\\n{\\r\\n  \\u201ctext\\u201d: \\u201cThe 38th NAACP Image Awards televised live on FOX in Hollywood, California, hosted by LL Cool J.\\u201d\\r\\n  \\u201corganizations\\u201d: [\\u201cFOX in\\u201d],\\r\\n  \\u201clocations\\u201d: [\\u201cHollywood, California\\u201d]\\r\\n}\\r\\n\\r\\n(LLM Response)\\r\\n{\\r\\n  \\u201cpeople\\u201d: [\\u201cLL Cool J\\u201d]\\r\\n  \\u201corganizations\\u201d: [\\u201cFOX\\u201d, \\u201cNAACP\\u201d],\\r\\n  \\u201clocations\\u201d: [\\u201cHollywood, California\\u201d]\\r\\n}\\r\\n\\r\\n<\\\/xmp>\\n\\t\\t\\t\\t<\\\/code>\\n\\t\\t\\t<\\\/pre>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjA1MzE2MGMiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD5PdXIgbW9kaWZpZWQgTHVjaWxsZSBFVEwgcGlwZWxpbmUgdXNlZCBvbmx5IHR3byBtb2RlbHMuIEZpcnN0LCBDb3JlTkxQIGV4dHJhY3RlZCBlbnRpdHkgbmFtZXMsIGFzIHVzdWFsLiBUaGVuLCB3ZSB1c2VkIGdlbW1hMyBhZ2FpbiwgYnV0IHdpdGggYSBuZXcgc3lzdGVtIHByb21wdCBhbmQgYSBkaWZmZXJlbnQgUHJvbXB0T2xsYW1hIGNvbmZpZ3VyYXRpb24uIE5vdywgZ2VtbWEzIHdhcyBpbnN0cnVjdGVkIHRvIFx1MjAxY2VkaXRcdTIwMWQgdGhlIHJlc3VsdHMgZnJvbSBDb3JlTkxQIGFzIG5lZWRlZC4gVGhlIHN0YWdlXHUyMDE5cyBjb25maWd1cmF0aW9uIGVuc3VyZWQgdGhlIHJlcXVlc3QgaW5jbHVkZWQgdGhlIHNvdXJjZSB0ZXh0IDxlbT5hbmQ8XC9lbT4gdGhlIHBlb3BsZSwgb3JnYW5pemF0aW9uLCBhbmQgbG9jYXRpb24gbmFtZXMgZXh0cmFjdGVkIGJ5IENvcmVOTFAuIChUaGlzIHdhcyB0aGUgb25seSBwaXBlbGluZSB3aGVyZSBhbiBMTE0gd2FzIHByb3ZpZGVkIHJlc3VsdHMgZnJvbSBhIHByZXZpb3VzIG1vZGVsLikgVG9nZXRoZXIsIHRoZSBtb2RlbHMgaGFkIHRoZSBmb2xsb3dpbmcgc2NvcmVzOjxcL3A+In0sImVsZW1lbnRzIjpbXSwid2lkZ2V0VHlwZSI6InRleHQtZWRpdG9yIn0=\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-f9cb6d5 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\\\" data-id=\\\"f9cb6d5\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"image.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<img width=\\\"653\\\" height=\\\"384\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp3.png\\\" class=\\\"attachment-large size-large wp-image-30242\\\" alt=\\\"\\\" srcset=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp3.png 653w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp3-300x176.png 300w\\\" sizes=\\\"(max-width: 653px) 100vw, 653px\\\" \\\/>\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjAzN2U4MWIiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD5VbmZvcnR1bmF0ZWx5LCB0aGlzIGFwcHJvYWNoIHdhcyBhY3R1YWxseSBsZXNzIHBlcmZvcm1hbnQsIHdpdGggYW4gRjEgc2NvcmUgbG93ZXIgdGhhbiBDb3JlTkxQIG9yIGdlbW1hMyBvcGVyYXRpbmcgaW5kaXZpZHVhbGx5LiBJbnN0ZWFkIG9mIGZpbmRpbmcgYSB3YXkgdG8gaW5jbHVkZSB0aG9zZSBcdTIwMWN1bmlxdWVcdTIwMWQgZ29sZCB3b3JkcywgaXQgbG9va3MgbGlrZSB0aGUgTExNIGhhZCBhbiBpbmNsaW5hdGlvbiB0byBkZWxldGUgdGhlIG5hbWVzIG91dHB1dCBieSBDb3JlTkxQLiAoSW4gYSBsYXRlciBjaGFydCwgeW91XHUyMDE5bGwgc2VlIHRoZSB0b3RhbCBudW1iZXIgb2Ygd29yZHMgb3V0cHV0IGluIHRoaXMgXHUyMDFjdHdvIHBhc3NcdTIwMWQgcGlwZWxpbmUgaXMgdmVyeSBzaW1pbGFyIHRvIHRoZSBudW1iZXIgb3V0cHV0IGJ5IGdlbW1hMyBhbG9uZS4pIFdoaWxlIHRoZXJlIGFyZSBjZXJ0YWlubHkgYSB2YXJpZXR5IG9mIHdheXMgdG8gdHdlYWsgdGhlIHBpcGVsaW5lLCBpdCBzZWVtZWQgd2Ugd2VyZW5cdTIwMTl0IGdvaW5nIHRvIG9idGFpbiB0aGUgcmVzdWx0cyB3ZSB3ZXJlIGxvb2tpbmcgZm9yIHdpdGggdGhpcyBhcHByb2FjaC48XC9wPiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0ZXh0LWVkaXRvciJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-e779735 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"e779735\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<h2 class=\\\"elementor-heading-title elementor-size-medium\\\">Experiment 4: Alternate Model<\\\/h2>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6ImM2ZjRkNzYiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD5MYXN0bHksIHdlIHdhbnRlZCB0byBtZWFzdXJlIHRoZSBwZXJmb3JtYW5jZSBvZiBhbiBhbHRlcm5hdGUgTExNLiBXZSBjcmVhdGVkIGEgcGlwZWxpbmUgc2ltaWxhciB0byB0aGUgZmlyc3QgZXhwZXJpbWVudCwgYnV0IFByb21wdE9sbGFtYSB1c2VkIGRlZXBzZWVrLXIxIGluc3RlYWQgb2YgZ2VtbWEzLiBkZWVwc2Vlay1yMSBpcyBhIFx1MjAxY3JlYXNvbmluZ1x1MjAxZCBtb2RlbCwgd2hpY2ggY291bGQgcG90ZW50aWFsbHkgeWllbGQgZGlmZmVyZW50IHJlc3VsdHMuIFRoZSB2YXJpYW50IHdlIHVzZWQsIGRlZXBzZWVrLXIxOjE0YiwgaGFkIDE0IGJpbGxpb24gcGFyYW1ldGVycywgYW5kIHdhcyByb3VnaGx5IDkgR0IgaW4gc2l6ZS4gVGhpcyBtYWRlIGl0IHNsaWdodGx5IGxhcmdlciB0aGFuIGdlbW1hMzoxMmIsIHRoZSBcdTIwMWNsYXJnZVx1MjAxZCBtb2RlbCB1c2VkIGluIHRoZSBmaXJzdCBwaXBlbGluZS48XC9wPiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0ZXh0LWVkaXRvciJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-77fbeaf flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\\\" data-id=\\\"77fbeaf\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"image.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<img width=\\\"446\\\" height=\\\"274\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4.png\\\" class=\\\"attachment-large size-large wp-image-30244\\\" alt=\\\"\\\" srcset=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4.png 446w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4-300x184.png 300w\\\" sizes=\\\"(max-width: 446px) 100vw, 446px\\\" \\\/>\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<div class=\\\"elementor-element elementor-element-2ef2fd8 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\\\" data-id=\\\"2ef2fd8\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"image.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<img width=\\\"448\\\" height=\\\"275\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4_gold.png\\\" class=\\\"attachment-large size-large wp-image-30243\\\" alt=\\\"\\\" srcset=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4_gold.png 448w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4_gold-300x184.png 300w\\\" sizes=\\\"(max-width: 448px) 100vw, 448px\\\" \\\/>\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjljZDgwYTQiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD5UaGUgXHUyMDFjdW5pcXVlXHUyMDFkIGNvdW50IG9ubHkgY29uc2lkZXJlZCB0aGVzZSB0aHJlZSBtb2RlbHMsIHNvIHRoZSByZXN1bHRzIGFyZW5cdTIwMTl0IGRpcmVjdGx5IGNvbXBhcmFibGUgdG8gdGhlIHNhbWUgY2hhcnQgZnJvbSBFeHBlcmltZW50IDIuPFwvcD48cD5XZSBjYW4gc2VlIHRoYXQgZGVlcHNlZWstcjFcdTIwMTlzIHBlcmZvcm1hbmNlIHdhcyByb3VnaGx5IGluIGxpbmUgd2l0aCBnZW1tYTNcdTIwMTlzIHBlcmZvcm1hbmNlIGZyb20gZWFybGllciwgd2l0aCBhbiBGMSBzY29yZSBvZiByb3VnaGx5IDAuNy48XC9wPjxwPkFnYWluLCB3ZSBub3RpY2VkIHRoYXQgQ29yZU5MUCBhbmQgdGhlIExMTSB3ZXJlIGVhY2ggcGlja2luZyB1cCBvbiBtYW55IGdvbGQgd29yZHMgdGhhdCB0aGUgb3RoZXIgbW9kZWxzIHdlcmVuXHUyMDE5dC4gV2Ugc3RpbGwgd2FudGVkIHRvIGZpbmQgYSB3YXkgdG8gY2FwdHVyZSBhcyBtYW55IGdvbGQgd29yZHMgYXMgcG9zc2libGUuIFNvLCBpbnN0ZWFkIG9mIHJ1bm5pbmcgYW5vdGhlciBwaXBlbGluZSwgd2UgZGVjaWRlZCB0byBqdXN0IGNhbGN1bGF0ZSB0aGUgc2NvcmVzIGFzc29jaWF0ZWQgd2l0aCBjb21iaW5pbmcgdGhlIG91dHB1dHMgb2YgZXZlcnkgbW9kZWw6PFwvcD4ifSwiZWxlbWVudHMiOltdLCJ3aWRnZXRUeXBlIjoidGV4dC1lZGl0b3IifQ==\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-6378376 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\\\" data-id=\\\"6378376\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"image.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<img width=\\\"936\\\" height=\\\"490\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4_agg.png\\\" class=\\\"attachment-large size-large wp-image-30245\\\" alt=\\\"\\\" srcset=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4_agg.png 936w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4_agg-300x157.png 300w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/exp4_agg-768x402.png 768w\\\" sizes=\\\"(max-width: 936px) 100vw, 936px\\\" \\\/>\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6ImE3NmFlZmYiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD5BcyB5b3UgY291bGQgaW1hZ2luZSwgd2UgZ290IGhpZ2hlciByZWNhbGwgYXQgdGhlIGNvc3Qgb2YgcmVkdWNlZCBwcmVjaXNpb24uIFdlIHBpY2tlZCB1cCBvbiBtb3JlIG9mIHRoZSBnb2xkIHdvcmRzLCBidXQgbGVzcyBvZiB0aGUgd29yZHMgbGlzdGVkIHdlcmUgYWN0dWFsbHkgZ29sZCB3b3JkcyBmcm9tIHRoZSBvcmlnaW5hbCBkYXRhc2V0LiBJbnRlcmVzdGluZ2x5LCB0aGUgRjEgc2NvcmUgcmVtYWluZWQgcm91Z2hseSB0aGUgc2FtZSwgZGVzcGl0ZSBwcm9ub3VuY2VkIHNoaWZ0cyBpbiBwcmVjaXNpb24gYW5kIHJlY2FsbC48XC9wPjxwPklmIHlvdVx1MjAxOXJlIGxvb2tpbmcgdG8gcnVuIGVuaGFuY2VkIHNlYXJjaGVzIG9uIHlvdXIgZG9jdW1lbnRzLCBhIGhpZ2hlciByZWNhbGwgd2lsbCBoZWxwIGVuc3VyZSB5b3UgZG9uXHUyMDE5dCBtaXNzIG91dCBvbiBhbnkgbmFtZXMuIEJ1dCwgaWYgeW91XHUyMDE5cmUgbG9va2luZyB0byBydW4gYWdncmVnYXRpb25zIG9yIGZhY2V0cyBvbiB0aGUgZXh0cmFjdGVkIGVudGl0aWVzLCB0aGVzZSBleHRyYSBub24tZ29sZCBlbnRyaWVzIGNvdWxkIHVuZGVybWluZSB0aGUgcXVhbGl0eSBvZiB5b3VyIGluc2lnaHRzLjxcL3A+PHA+QWdhaW4sIHdlIG5vdGljZWQgdGhhdCBDb3JlTkxQIGFuZCB0aGUgTExNIHdlcmUgZWFjaCBwaWNraW5nIHVwIG9uIG1hbnkgZ29sZCB3b3JkcyB0aGF0IHRoZSBvdGhlciBtb2RlbHMgd2VyZW5cdTIwMTl0LiBXZSBzdGlsbCB3YW50ZWQgdG8gZmluZCBhIHdheSB0byBjYXB0dXJlIGFzIG1hbnkgZ29sZCB3b3JkcyBhcyBwb3NzaWJsZS4gU28sIGluc3RlYWQgb2YgcnVubmluZyBhbm90aGVyIHBpcGVsaW5lLCB3ZSBkZWNpZGVkIHRvIGp1c3QgY2FsY3VsYXRlIHRoZSBzY29yZXMgYXNzb2NpYXRlZCB3aXRoIGNvbWJpbmluZyB0aGUgb3V0cHV0cyBvZiBldmVyeSBtb2RlbDo8XC9wPiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0ZXh0LWVkaXRvciJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-8908134 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"8908134\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<h2 class=\\\"elementor-heading-title elementor-size-medium\\\">Pulling The Data Together<\\\/h2>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6Ijc4MTA2YTkiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD5MYXN0bHksIGhlcmUgYXJlIHNvbWUgaGlnaGVyIGxldmVsIHJlc3VsdHMgY29tcGFyaW5nIGFsbCBvZiB0aGUgbW9kZWxzLjxcL3A+In0sImVsZW1lbnRzIjpbXSwid2lkZ2V0VHlwZSI6InRleHQtZWRpdG9yIn0=\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-9cdbd70 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\\\" data-id=\\\"9cdbd70\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"image.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<img width=\\\"775\\\" height=\\\"484\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_f1.png\\\" class=\\\"attachment-large size-large wp-image-30248\\\" alt=\\\"\\\" srcset=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_f1.png 775w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_f1-300x187.png 300w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_f1-768x480.png 768w\\\" sizes=\\\"(max-width: 775px) 100vw, 775px\\\" \\\/>\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<div class=\\\"elementor-element elementor-element-1ec62f8 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\\\" data-id=\\\"1ec62f8\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"image.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<img width=\\\"790\\\" height=\\\"531\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_nowords.png\\\" class=\\\"attachment-large size-large wp-image-30247\\\" alt=\\\"\\\" srcset=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_nowords.png 790w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_nowords-300x202.png 300w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_nowords-768x516.png 768w\\\" sizes=\\\"(max-width: 790px) 100vw, 790px\\\" \\\/>\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<div class=\\\"elementor-element elementor-element-f0aea71 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-image\\\" data-id=\\\"f0aea71\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"image.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<figure class=\\\"wp-caption\\\">\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<img width=\\\"745\\\" height=\\\"459\\\" src=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_meanlatency.png\\\" class=\\\"attachment-large size-large wp-image-30246\\\" alt=\\\"\\\" srcset=\\\"https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_meanlatency.png 745w, https:\\\/\\\/kmwllc.com\\\/wp-content\\\/uploads\\\/2025\\\/10\\\/summary_meanlatency-300x185.png 300w\\\" sizes=\\\"(max-width: 745px) 100vw, 745px\\\" \\\/>\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<figcaption class=\\\"widget-image-caption wp-caption-text\\\">CPU \\\/ GPU: Apple M1 Pro   RAM: 16 GB.<\\\/figcaption>\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/figure>\\n\\t\\t\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<div class=\\\"elementor-element elementor-element-112f0be flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"112f0be\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<h1 class=\\\"elementor-heading-title elementor-size-large\\\">Conclusion<\\\/h1>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6ImI1NGZkYzUiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJlZGl0b3IiOiI8cD5PdmVyYWxsLCBpdCBsb29rcyBsaWtlIHRyYWRpdGlvbmFsIE5MUCBtb2RlbHMgYXJlIHN0aWxsIHZhbHVhYmxlLCBldmVuIGFzIExMTXMgYXJlIGZyZXF1ZW50bHkgdG91dGVkIGFzIHRoZSBzb2x1dGlvbiB0byBhbGwgb2YgbGlmZVx1MjAxOXMgcHJvYmxlbXMuIENvcmVOTFAgZ2VuZXJhbGx5IGxlZCB0aGUgd2F5IHdpdGggdGhlIGhpZ2hlc3QgRjEgc2NvcmVzIGFuZCBvbmx5IGEgZnJhY3Rpb24gb2YgdGhlIExMTXNcdTIwMTkgaGlnaCBsYXRlbmN5LiBCdXQgc29tZSBvZiB0aGUgTExNcyB3ZSB0ZXN0ZWQgd2VyZSBzdGlsbCB2ZXJ5IHN0cm9uZyBhbHRlcm5hdGl2ZXMuIFRoZXkgaGFkIGhpZ2ggRjEgc2NvcmVzIGFuZCBwaWNrZWQgdXAgb24gc29tZSB3b3JkcyB0aGF0IENvcmVOTFAgZGlkblx1MjAxOXQuIE9wZW5OTFAgZGlkIHVuZGVycGVyZm9ybSwgYnV0IHdlIHdlcmUgdXNpbmcgaXRzIHByZXRyYWluZWQgbW9kZWxzLiBUcmFpbmluZyBhIGN1c3RvbSBtb2RlbCB3aXRoIE9wZW5OTFBcdTIwMTlzIGFyY2hpdGVjdHVyZSBjb3VsZCB5aWVsZCBpbXByb3ZlZCByZXN1bHRzLjxcL3A+PHA+VGhvdWdoIENvcmVOTFAgb3V0cGVyZm9ybWVkLCBMTE1zIGNvdWxkIHN0aWxsIHBsYXkgYSB2aXRhbCByb2xlIGluIG1hbnkgZW50aXR5IGV4dHJhY3Rpb24gc29sdXRpb25zLCBhcyB0aGV5IGFyZSBleHRyZW1lbHkgdmVyc2F0aWxlIGFuZCByZXF1aXJlIG1pbmltYWwgc2V0dXAuIElmIHlvdSBkb25cdTIwMTl0IGhhdmUgdGhlIHRpbWUgdG8gZmluZCBhIHRyYWluaW5nIGRhdGFzZXQsIGNsZWFuc2UgaXQsIGFuZCB0aGVuIHRyYWluIGFuZCBldmFsdWF0ZSBhIG1vZGVsLCBhbiBMTE0gaXMgY2VydGFpbmx5IGEgdmlhYmxlIG9wdGlvbi4gQWRkaXRpb25hbGx5LCBhbiBMTE0gY291bGQgaGFuZGxlIGRhdGEgaW4gYSB2YXJpZXR5IG9mIGxhbmd1YWdlcyB3aXRob3V0IGFueSBhZGRpdGlvbmFsIGNvbmZpZ3VyYXRpb24gb3IgdHJhaW5pbmcgbmVlZGVkLiBJZiBvdXIgZGF0YSB3YXMgaW4gbXVsdGlwbGUgbGFuZ3VhZ2VzLCB3ZSB3b3VsZCBoYXZlIGhhZCB0byBjb21wbGV0ZWx5IG92ZXJoYXVsIG91ciBwaXBlbGluZSB0byBzdXBwb3J0IHRoaXMgZGF0YS48XC9wPjxwPkFzIHN1Y2gsIGFueSBlbnRpdHkgZXh0cmFjdGlvbiBzb2x1dGlvbiB5b3UgYnVpbGQgc2hvdWxkIGJlIHRhaWxvcmVkIHRvIHlvdXIgdXNlIGNhc2UuIFdoaWxlIHlvdSBjYW5cdTIwMTl0IHJlYWxseSBnbyB3cm9uZyB3aXRoIGEgdHJhZGl0aW9uYWwgbW9kZWwsIHlvdSBtYXkgd2FudCB0byBjb25zaWRlciBpbnRlZ3JhdGluZyBhbiBMTE0gaW50byB5b3VyIHByb2Nlc3MuIEFyZSB5b3VyIGRvY3VtZW50cyBpbiBtdWx0aXBsZSBsYW5ndWFnZXM\\\/IERvIHRoZXkgaGF2ZSB2ZXJ5IGxvbmcgcGllY2VzIG9mIHRleHQ\\\/IEhvdyBtYW55IGRvY3VtZW50cyBkbyB5b3UgaGF2ZT8gSG93IG11Y2ggY29tcHV0ZSBpcyBhdmFpbGFibGUgdG8geW91PyBZb3VcdTIwMTlsbCBoYXZlIHRvIHRha2UgYSBob2xpc3RpYyBhcHByb2FjaCB0byBkZXNpZ25pbmcgeW91ciBzb2x1dGlvbi48XC9wPjxwPkJhc2VkIG9uIG91ciBmaW5kaW5ncywgZXZlbiBpbiBhIHdvcmxkIGZpbGxlZCB3aXRoIExMTXMsIGl0IGxvb2tzIGxpa2UgdHJhZGl0aW9uYWwgbW9kZWxzIHN0aWxsIGhhdmUgYSBwbGFjZSBpbiBhZGRyZXNzaW5nIGNsYXNzaWMgTkxQIHByb2JsZW1zLjxcL3A+In0sImVsZW1lbnRzIjpbXSwid2lkZ2V0VHlwZSI6InRleHQtZWRpdG9yIn0=\\\"]\\t\\t\\t<\\\/div>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t<div class=\\\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-fe57f53\\\" data-id=\\\"fe57f53\\\" data-element_type=\\\"column\\\" data-e-type=\\\"column\\\">\\n\\t\\t\\t<div class=\\\"elementor-widget-wrap\\\">\\n\\t\\t\\t\\t\\t\\t\\t<\\\/div>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t<div class=\\\"elementor-column elementor-col-33 elementor-top-column elementor-element elementor-element-82d929d\\\" data-id=\\\"82d929d\\\" data-element_type=\\\"column\\\" data-e-type=\\\"column\\\">\\n\\t\\t\\t<div class=\\\"elementor-widget-wrap elementor-element-populated\\\">\\n\\t\\t\\t\\t\\t\\t<div class=\\\"elementor-element elementor-element-44d87a7 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"44d87a7\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<div class=\\\"title-h6 elementor-heading-title elementor-size-small\\\">Share Post<\\\/div>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjVlYjgwZjMiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJwaW50ZXJlc3QiOiIiLCJ0dW1ibHIiOiIiLCJ0ZWxlZ3JhbSI6IiIsIndoYXRzYXBwIjoiIiwidmliZXIiOiIiLCJ4aW5nIjoiIiwiaWNvbnNfY29sb3IiOiIjMDBERUZGIn0sImVsZW1lbnRzIjpbXSwid2lkZ2V0VHlwZSI6InRoZWdlbS1zb2NpYWwtc2hhcmluZyJ9\\\"]\\t\\t<div class=\\\"elementor-element elementor-element-aa98076 flex-horizontal-align-default flex-horizontal-align-tablet-default flex-horizontal-align-mobile-default flex-vertical-align-default flex-vertical-align-tablet-default flex-vertical-align-mobile-default elementor-widget elementor-widget-heading\\\" data-id=\\\"aa98076\\\" data-element_type=\\\"widget\\\" data-e-type=\\\"widget\\\" data-widget_type=\\\"heading.default\\\">\\n\\t\\t\\t\\t<div class=\\\"elementor-widget-container\\\">\\n\\t\\t\\t\\t\\t<div class=\\\"title-h6 elementor-heading-title elementor-size-small\\\">More From the KMW Blog<\\\/div>\\t\\t\\t\\t<\\\/div>\\n\\t\\t\\t\\t<\\\/div>\\n\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6ImIxZDcyODgiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJ0aGVnZW1fZWxlbWVudG9yX3ByZXNldCI6ImNvbXBhY3QtdGlueS0yIiwic291cmNlIjpbInBvc3RzIl0sInNob3dfc2VwYXJhdG9yIjoiIiwic2hvd19jb21tZW50cyI6IiIsInJlYWRtb3JlX2J1dHRvbl90ZXh0IjoiUmVhZCBNb3JlIiwibG9hZG1vcmVfYnV0dG9uX3RleHQiOiJMb2FkIE1vcmUiLCJjYXB0aW9uX2NhdGVnb3JpZXNfaW5fdGV4dCI6ImluICIsImNhcHRpb25fYXV0aG9yX2J5X3RleHQiOiJCeSIsInNvdXJjZV90eXBlIjoiY3VzdG9tIiwiZXhjbHVkZV9ibG9nX3Bvc3RzX3R5cGUiOiJjdXJyZW50IiwicGFnaW5hdGlvbl90eXBlIjoibnVtYmVycyJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJ0aGVnZW0tYmxvZ2xpc3QifQ==\\\"]\\t\\t\\t<\\\/div>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t<\\\/div><\\\/div>\\r\\n\\t\\t<\\\/section>\\r\\n\\t\\t\\t\\t<section class=\\\"elementor-section elementor-top-section elementor-element elementor-element-0ee5fe2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\\\" data-id=\\\"0ee5fe2\\\" data-element_type=\\\"section\\\" data-e-type=\\\"section\\\">\\r\\n\\t\\t\\t\\t\\t\\t<div class=\\\"elementor-container elementor-column-gap-thegem\\\"><div class=\\\"elementor-row\\\">\\r\\n\\t\\t\\t\\t\\t<div class=\\\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-499f8b9\\\" data-id=\\\"499f8b9\\\" data-element_type=\\\"column\\\" data-e-type=\\\"column\\\">\\n\\t\\t\\t<div class=\\\"elementor-widget-wrap elementor-element-populated\\\">\\n\\t\\t\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjkxYTJlZGEiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJwcmV2X2xhYmVsIjoiUHJldmlvdXMgUG9zdCIsIm5leHRfbGFiZWwiOiJOZXh0IFBvc3QiLCJzaG93X2JvcmRlcnMiOiIiLCJ0aXRsZV90eXBvZ3JhcGh5X3R5cG9ncmFwaHkiOiJjdXN0b20iLCJ0aXRsZV90eXBvZ3JhcGh5X2ZvbnRfc2l6ZSI6eyJ1bml0IjoicHgiLCJzaXplIjoxNCwic2l6ZXMiOltdfSwidGl0bGVfdHlwb2dyYXBoeV9mb250X3dlaWdodCI6IjcwMCIsIl9fZ2xvYmFsc19fIjp7ImFycm93X2NvbG9yIjoiZ2xvYmFsc1wvY29sb3JzP2lkPXByaW1hcnkiLCJsYWJlbF9jb2xvciI6Imdsb2JhbHNcL2NvbG9ycz9pZD1zZWNvbmRhcnkifSwiYXJyb3ciOiJmYSBmYS1jYXJldC1sZWZ0Iiwic2hvd19hcnJvdyI6IiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJwb3N0LW5hdmlnYXRpb24ifQ==\\\"]\\t\\t\\t<\\\/div>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t<\\\/div><\\\/div>\\r\\n\\t\\t<\\\/section>\\r\\n\\t\\t\",\"scripts\":[],\"styles\":[\"e-swiper\",\"thegem-bloglist\"]}}"]},"jetpack_featured_media_url":"https:\/\/kmwllc.com\/wp-content\/uploads\/2025\/10\/blogpost_entityex.png","menu_order":0,"_links":{"self":[{"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/posts\/30125","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/users\/14"}],"replies":[{"embeddable":true,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/comments?post=30125"}],"version-history":[{"count":39,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/posts\/30125\/revisions"}],"predecessor-version":[{"id":30281,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/posts\/30125\/revisions\/30281"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/media\/30252"}],"wp:attachment":[{"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/media?parent=30125"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/categories?post=30125"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/tags?post=30125"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}