{"id":26175,"date":"2020-03-10T04:38:00","date_gmt":"2020-03-09T23:38:00","guid":{"rendered":"http:\/\/3.85.220.248\/?p=26175"},"modified":"2024-05-01T07:26:03","modified_gmt":"2024-05-01T02:26:03","slug":"understanding-scoring-through-examples","status":"publish","type":"post","link":"https:\/\/kmwllc.com\/index.php\/2020\/03\/10\/understanding-scoring-through-examples\/","title":{"rendered":"Understanding Scoring Through Examples"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\r\n\t\t\t\t\t\t
\r\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t
\r\n\t\t\t\t\t\t

<\/p>\n

<\/p>\n

<\/p>\n

The built-in scoring mechanism in Elasticsearch and Solr can seem mysterious to beginners and experienced practitioners alike. Instead of delving into the mathematical definitions of TF-IDF and BM25, this article will help you develop an intuitive understanding of these metrics by walking you through a series of simple examples. Each example consists of a query and list of several indexed documents. As you read along, try to guess which document comes up on top for each query. In each case, we will examine why that particular document gets the highest score and we\u2019ll extract the general principle behind this behavior. A set of six examples will be followed by an extra credit section focusing on more advanced topics. Along with illustrating all of the key behaviors of BM25, our examples will touch on some of the gotchas around scoring in cluster scenario, where shards and replicas come into play. This article aims to teach you, in a short time and without any math, everything you\u2019ll ever need to know about scoring. Having a solid understanding of scoring will prepare you to better diagnose relevance problems and improve relevance in real-world applications.<\/p>\n

<\/p>\n

<\/p>\n

Query 1: dog<\/h4>\n

<\/p>\n

<\/p>\n

Let\u2019s say I search for \u201cdog\u201d and there are only three documents in my index, as shown below. Which one of these documents is going to come up on top?<\/p>\n

<\/p>\n

<\/p>\n

Doc 1: \"dog\"\n\nDoc 2: \"dog dog\"\n\nDoc 3: \"dog dog dog<\/span><\/pre>\n
<\/p>\n
<\/p>\n
If you\u2019re not quite sure, that\u2019s good, because I haven\u2019t given you enough of the context to know the answer. All the queries in this article were tested in Elasticsearch 7.4 where BM25 is the default scoring algorithm and its parameters are set as k=1.2 and b=0.7. (Please ignore that if it\u2019s meaningless to you.) In most of the examples, we\u2019ll assume the documents have a single text field called \u201ctitle\u201d that uses the standard analyzer:<\/p>\n
<\/p>\n
<\/p>\n
\"mappings\": {\n    \"properties\": {\n      \"title\": {\n        \"type\": \"text\",\n        \"analyzer\": \"standard\"\n      }\n    }\n  }<\/span><\/pre>\n
<\/p>\n
<\/p>\n
For the most part, we\u2019ll be doing simple\u00a0match<\/strong>\u00a0queries against the title field. Recall that the default boolean operator in Elasticsearch is\u00a0OR<\/strong>. Here\u2019s what our \u201cdog\u201d query looks like:<\/p>\n
<\/p>\n
<\/p>\n
GET test\/_search\n{\n  \"query\": {\n      \"match\": {\n          \"title\": \"dog\"\n      } \n  }\n}<\/span><\/pre>\n
<\/p>\n
<\/p>\n
With those details out of the way, are you ready to tell me which one of those three documents (\u201cdog,\u201d \u201cdog dog,\u201d and \u201cdog dog dog\u201d) is going to get the highest score?<\/p>\n
<\/p>\n
<\/p>\n
Here are the results:<\/p>\n
<\/p>\n
<\/p>\n
\n\n\n\n\n\n\n
ID<\/strong><\/td>\nTitle<\/strong><\/td>\nScore<\/strong><\/td>\n<\/tr>\n
1<\/td>\ndog<\/td>\n0.167<\/td>\n<\/tr>\n
2<\/td>\ndog dog<\/td>\n0.183<\/td>\n<\/tr>\n
3<\/td>\ndog dog dog<\/td>\n0.189<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
<\/p>\n
<\/p>\n
Doc 3 gets the highest score because it has contains the highest number of tokens that match the query term. Another way to say this is that Doc 3 has the highest\u00a0term frequency<\/strong>\u00a0for the term in question. Of course if we were using the keyword analyzer, only Doc 1 would have matched the query, but we\u2019re using the standard analyzer which breaks the title into multiple tokens. The moral of this story is that, from a scoring perspective at least,<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
High term frequency is good.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
Before we move on, take a moment to compare Doc 1 and Doc 3. Notice that although Doc 3 has three times the term frequency for \u201cdog\u201d as Doc 1, its score isn\u2019t three times as high. So while higher term frequency gives a higher score, its impact is not multiplicative.<\/p>\n
<\/p>\n
<\/p>\n
Query 2: dog dog cat<\/h4>\n
<\/p>\n
<\/p>\n
Now I\u2019m searching for \u201cdog dog cat\u201d and there are only two documents in my index:<\/p>\n
<\/p>\n
<\/p>\n
Doc 1: \"cat\"\nDoc 2: \"dog\"<\/span><\/pre>\n
<\/p>\n
<\/p>\n
Which one of these is going to come up on top? Or are they going to be tied?<\/p>\n
<\/p>\n
<\/p>\n
In fact, \u201cdog\u201d is the winner here:<\/p>\n
<\/p>\n
<\/p>\n
\n\n\n\n\n\n
ID<\/strong><\/td>\nTitle<\/strong><\/td>\nScore<\/strong><\/td>\n<\/tr>\n
1<\/td>\ncat<\/td>\n0.6<\/td>\n<\/tr>\n
2<\/td>\ndog<\/td>\n1.3<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
<\/p>\n
<\/p>\n
Why does \u201cdog\u201d get twice the score of \u201ccat?\u201d The lesson here is that<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
Scores for each query term are summed.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
Our query has two instances of \u201cdog.\u201d The score for the whole query is the sum of the scores for each term. Each instance of \u201cdog\u201d in our query is going to match the \u201cdog\u201d in Doc 2 and contribute roughly 0.6 to the score, for a total of 1.3. Using the standard analyzer, query terms aren\u2019t deduplicated, so each instance of \u201cdog\u201d is treated separately. Doc 1 doesn\u2019t get a similar advantage because our query only contains one instance of \u201ccat.\u201d<\/p>\n
<\/p>\n
<\/p>\n
Query 3: dog dog cat<\/h4>\n
<\/p>\n
<\/p>\n
Now I\u2019m executing the same query as before, \u201cdog dog cat,\u201d but my index is different. This time I have lots of \u201cdog\u201d documents:<\/p>\n
<\/p>\n
<\/p>\n
Doc 1: \"dog\"\nDoc 2: \"dog\"\nDoc 3: \"dog\"\nDoc 4: \"dog\"\nDoc 5: \"dog\"\nDoc 6: \"dog\"\nDoc 7: \"cat\"<\/span><\/pre>\n
<\/p>\n
<\/p>\n
What\u2019s going to happen now? Do the \u201cdog\u201d documents still win over \u201ccat\u201d because my query mentions \u201cdog\u201d twice? Here are the results:<\/p>\n
<\/p>\n
<\/p>\n
\n\n\n\n\n\n\n\n\n\n\n
ID<\/strong><\/td>\nTitle<\/strong><\/td>\nScore<\/strong><\/td>\n<\/tr>\n
1<\/td>\ndog<\/td>\n0.4<\/td>\n<\/tr>\n
2<\/td>\ndog<\/td>\n0.4<\/td>\n<\/tr>\n
3<\/td>\ndog<\/td>\n0.4<\/td>\n<\/tr>\n
4<\/td>\ndog<\/td>\n0.4<\/td>\n<\/tr>\n
5<\/td>\ndog<\/td>\n0.4<\/td>\n<\/tr>\n
6<\/td>\ndog<\/td>\n0.4<\/td>\n<\/tr>\n
7<\/td>\ncat<\/td>\n1.5<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
<\/p>\n
<\/p>\n
The results are different this time because the terms have different\u00a0document frequencies<\/strong>\u00a0than before. A term\u2019s document frequency is the number of documents in the index that contain the term. From a scoring perspective, low document frequency is good and high document frequency is bad. In this example, \u201ccat\u201d is a rare term in the index \u2014 it has low document frequency \u2014 so matches on that term help the score more than matches on \u201cdog,\u201d which is a common term. The lesson here is that:<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
Matches for rarer terms are better.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
If I want to tell the search engine that a common term is particularly important to me in a certain scenario, I can\u00a0boost<\/strong>\u00a0the term. If I had executed my query with a boost of 7 on \u201cdog,\u201d the dog documents would come up above the cat document. Here\u2019s how I\u2019d set that up:<\/p>\n
<\/p>\n
<\/p>\n
GET test\/_search\n{\n \"query\": {\n  \"query_string\": {\n   \"query\": \"dog^7 cat\",\n   \"fields\": [\"title\"]\n  } \n }\n}<\/span><\/pre>\n
<\/p>\n
<\/p>\n
Query 4: dog cat<\/h4>\n
<\/p>\n
<\/p>\n
In this example I\u2019m searching for \u201cdog cat,\u201d and I\u2019ve got three documents in my index: one with a lot of dogs, one with a lot of cats, and one with a single instance of dog and cat each, plus a lot of mats.<\/p>\n
<\/p>\n
<\/p>\n
Doc 1: \"dog dog dog dog dog dog dog\"\nDoc 2: \"cat cat cat cat cat cat cat\"\nDoc 3: \"dog cat mat mat mat mat mat\"<\/span><\/pre>\n
<\/p>\n
<\/p>\n
Which document comes up on top this time? Notice that in Doc 1 and Doc 2, every single term matches one of the query terms, whereas in Doc 3 there are five terms that don\u2019t match anything. So the results might be a little surprising:<\/p>\n
<\/p>\n
<\/p>\n
\n\n\n\n\n\n\n
ID<\/td>\nTitle<\/td>\nScore<\/td>\n<\/tr>\n
1<\/td>\ndog dog dog dog dog dog dog<\/td>\n0.88<\/td>\n<\/tr>\n
2<\/td>\ncat cat cat cat cat cat cat<\/td>\n0.88<\/td>\n<\/tr>\n
3<\/td>\ndog cat mat mat mat mat mat<\/td>\n0.94<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
<\/p>\n
<\/p>\n
Document 3 gets the highest score because it has matches for\u00a0both<\/em>\u00a0of the query terms, \u201cdog\u201d and \u201ccat.\u201d While Documents 1 and 2 have higher term frequency for \u201cdog\u201d and \u201ccat\u201d respectively, they each contain only one of the terms. The lesson is that<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
Matching more query terms is good.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
Query 5: dog<\/h4>\n
<\/p>\n
<\/p>\n
Now I\u2019ll search for \u201cdog\u201d and there are only two documents in my index:<\/p>\n
<\/p>\n
<\/p>\n
Doc 1: \"dog cat zebra\"\nDoc 2: \"dog cat<\/span><\/pre>\n
<\/p>\n
<\/p>\n
Both of these documents match my query, and both have some terms that don\u2019t match. Which document does the best? Here are there results:<\/p>\n
<\/p>\n
<\/p>\n
\n\n\n\n\n\n
ID<\/td>\nTitle<\/td>\nScore<\/td>\n<\/tr>\n
1<\/td>\ndog cat zebra<\/td>\n0.16<\/td>\n<\/tr>\n
2<\/td>\ndog cat<\/td>\n0.19<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
<\/p>\n
<\/p>\n
In this case, Document 2 does better because it is shorter. The thinking is that when a term occurs in a shorter document, we can be more confident that the term is significant to the document (or that the document is\u00a0about<\/em>\u00a0the term). When a term occurs in a longer document, we have less confidence that this occurrence is meaningful. The lesson here is that<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
Matches in shorter documents are better.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
Query 6: orange dog<\/h4>\n
<\/p>\n
<\/p>\n
Now let\u2019s consider a scenario that\u2019s a little more complicated than the previous ones. For the first time, our documents will have two fields, \u201ccolor\u201d and \u201ctype.\u201d<\/p>\n
<\/p>\n
<\/p>\n
Doc 1: {\"color\": \"brown\", \"type\": \"dog\"}\nDoc 2: {\"color\": \"brown\", \"type\": \"dog\"}\nDoc 3: {\"color\": \"brown\", \"type\": \"cat\"}\nDoc 4: {\"color\": \"orange\", \"type\": \"cat\"}<\/span><\/pre>\n
<\/p>\n
<\/p>\n
We\u2019re searching for \u201corange dog\u201d but as you can see, there are no orange dogs in the index. There are two brown dogs, a brown cat, and an orange cat. Which one is going to come up on top?<\/p>\n
<\/p>\n
<\/p>\n
I should mention that we\u2019re searching across both fields using a\u00a0multi_match<\/strong>\u00a0like this:<\/p>\n
<\/p>\n
<\/p>\n
GET test\/_search\n{\n  \"query\": {\n   \"multi_match\": {\n    \"query\": \"orange dog\", \n    \"fields\": [\"type\", \"color\"],\n    \"type\": \"most_fields\"\n  }\n }\n}<\/span><\/pre>\n
<\/p>\n
<\/p>\n
Here are the results:<\/p>\n
<\/p>\n
<\/p>\n
\n\n\n\n\n\n\n\n
ID<\/td>\nColor<\/td>\nType<\/td>\nScore<\/td>\n<\/tr>\n
1<\/td>\nbrown<\/td>\ndog<\/td>\n0.6<\/td>\n<\/tr>\n
2<\/td>\nbrown<\/td>\ndog<\/td>\n0.6<\/td>\n<\/tr>\n
3<\/td>\nbrown<\/td>\ncat<\/td>\nN\/A<\/td>\n<\/tr>\n
4<\/td>\norange<\/td>\ncat<\/td>\n1.2<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
<\/p>\n
<\/p>\n
This example hints at some of the unexpected behavior that can arise when we search across multiple fields. The search engine doesn\u2019t know which field is most important to us. If someone is searching for an orange dog, we might guess they\u2019re more interested in seeing dogs than seeing arbitrary things that happen to be orange. (\u201cOrange dog\u201d would be very strange query to enter if you meant \u201cshow me anything that\u2019s orange.\u201d) In this case, however, the color field is taking priority because \u201corange\u201d is a rare term within that field (there are 3 browns and only 1 orange). Within the type field, \u201cdog\u201d and \u201ccat\u201d have the same frequency. The orange cat comes up on top because a match for the rare term \u201corange\u201d is treated as more valuable than a match for \u201cdog.\u201d<\/p>\n
<\/p>\n
<\/p>\n
If we want to give more weight to the \u201ctype\u201d field we can boost it like this:<\/p>\n
<\/p>\n
<\/p>\n
GET test\/_search\n{\n  \"query\": {\n   \"multi_match\": {\n    \"query\": \"orange dog\", \n    \"fields\": [\u201ctype^2\", \"color\"],\n    \"type\": \"most_fields\"\n  }\n }\n}<\/span><\/pre>\n
<\/p>\n
<\/p>\n
With the boost applied, the \u201cbrown dog\u201d documents now score 1.3 and come up above the \u201corange cat.\u201d<\/p>\n
<\/p>\n
<\/p>\n
The lesson here is that searching across multiple fields can be tricky because the per-field scores are added together without concern for which fields are more important. To rectify this,<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
We can use boosting to express field priorities.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
Query 7: dog<\/h4>\n
<\/p>\n
<\/p>\n
Now we\u2019re moving into advanced territory although our query looks simpler than anything we\u2019ve seen before. We\u2019re searching for \u201cdog\u201d and our index has three identical \u201cdog\u201d documents:<\/p>\n
<\/p>\n
<\/p>\n
Doc 1: \"dog\"\nDoc 2: \"dog\"\nDoc 3: \"dog\"<\/span><\/pre>\n
<\/p>\n
<\/p>\n
Which one is going to come up on top? You might guess that these three documents, being identical, should get the same score. So let\u2019s take a moment to look at how ties are handled. When two documents have the same score, they\u2019ll be sorted by their internal Lucene doc id. This internal id is different from the value in the document\u2019s\u00a0_id<\/strong>\u00a0field, and it can differ even for the same document across replicas of a shard. If you really want ties to be broken in the same way regardless of which replica you hit, you can add a sort to your query, where you sort first by\u00a0_score<\/strong>\u00a0and then by a designated tiebreaker like _id or date.<\/p>\n
<\/p>\n
<\/p>\n
But this point about tiebreaking is only an aside. When I actually ran this query, the documents\u00a0didn\u2019t<\/em>\u00a0come back with identical scores!<\/p>\n
<\/p>\n
<\/p>\n
\n\n\n\n\n\n\n
ID<\/strong><\/td>\nTitle<\/strong><\/td>\nScore<\/strong><\/td>\n<\/tr>\n
1<\/td>\ndog<\/td>\n0.28<\/strong><\/td>\n<\/tr>\n
2<\/td>\ndog<\/td>\n0.18<\/td>\n<\/tr>\n
3<\/td>\ndog<\/td>\n0.18<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
<\/p>\n
<\/p>\n
How is it possible that identical documents would get different scores? The lesson in this example is that<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
Term statistics are measured per shard.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
Though I didn\u2019t state it explicitly at the beginning of the post, all our previous examples were using one shard. In the current example, however, I set up my index with two shards. Document 1 landed on Shard 1 while Documents 2 and 3 landed on Shard 2. Document 1 got a higher score because within Shard 1, \u201cdog\u201d is a rarer term \u2014 it only occurs once. Within Shard 2, \u201cdog\u201d is more common \u2014 it occurs twice. Here\u2019s how I set up the example:<\/p>\n
<\/p>\n
<\/p>\n
PUT \/test \n{ \"settings\": { \"number_of_shards\": 2 } }\n\nPUT \/test\/_doc\/1?routing=0\n{ \"title\" : \"dog\" } \n\nPUT \/test\/_doc\/2?routing=1\n{ \"title\" : \"dog\" } \n\nPUT \/test\/_doc\/3?routing=1\n{ \"title\" : \"dog\" } <\/span><\/pre>\n
<\/p>\n
<\/p>\n
If you\u2019re working with multiple shards and you want scores to be consistent regardless of which shard a document lives in, you can do a Distributed Frequency Search by added the following parameter to your query:\u00a0search_type=dfs_query_then_fetch<\/strong>. This tells Elasticsearch to retrieve term statistics from all the shards and combine them before computing the scores.<\/p>\n
<\/p>\n
<\/p>\n
But its also important to know that<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
Replicas of a shard may have different term statistics.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
This is a consequence of how deletion is handled in Lucene. Documents that are marked for deletion but not yet physically removed (when their segments are merged) still contribute to term statistics. When a document is deleted, all the replicas will immediately \u201cknow\u201d about the deletion, but they might not carry it out physically at the same time, so they might end up with different term statistics. To reduce the impact of this, you can specify a user or session ID in the shard copy\u00a0preference<\/strong>\u00a0parameter. This encourages Elasticsearch to route requests from the same user to the same replicas, so that, for example, a user will not notice scoring discrepancies when issuing the same query multiple times.<\/p>\n
<\/p>\n
<\/p>\n
Its also important to know that, from a scoring perspective,<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
Document updates behave like insertions, until segments are merged.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
When you update a document in Lucene, a new version of it is written to disk and the old version is marked for deletion. But the old version continues to contribute to term statistics until it is physically deleted.<\/p>\n
<\/p>\n
<\/p>\n
In the example below, I create a \u201cdog cat\u201d document and then I update its contents to be \u201cdog zebra.\u201d Immediately after the update, if I query for \u201cdog\u201d and look at the explain output, Elasticsearch tells me there are\u00a0two\u00a0<\/em>documents containing the term \u201cdog.\u201d The number goes down to one after I do a\u00a0_forcemerge<\/strong>. The moral: if you\u2019re doing relevancy tuning in Elasticsearch, looking closely at scores, and also updating documents at the same time, be sure to run a _forcemerge after your updates, or else rebuild the index entirely.<\/p>\n
<\/p>\n
<\/p>\n
PUT test\/_doc\/1\n{ \"title\": \"dog cat\" }\n\nGET test\/_search?format=yaml\n{ \"query\" : { \"match\" : { \"title\": \"dog\" } }, \n  \"explain\": true }\n\nPUT test\/_doc\/1?refresh\n{ \"title\": \"dog zebra\" }\n\nGET test\/_search?format=yaml\n{ \"query\" : { \"match\" : { \"title\": \"dog\" } }, \n  \"explain\": true }\n\nPOST test\/_forcemerge\n\nGET test\/_search?format=yaml\n{ \"query\" : { \"match\" : { \"title\": \"dog\" } }, \n  \u201cexplain\": true }<\/span><\/pre>\n
<\/p>\n
<\/p>\n
Query 8: dog cat<\/h4>\n
<\/p>\n
<\/p>\n
Now let\u2019s take another look at Query 4, where we searched for \u201cdog cat\u201d and we found that a document containing both terms did better than documents with lots of instances of one term or the other. In Query 4, we were searching the title field, the only field available. Here, we\u2019ve got\u00a0two<\/em>\u00a0fields, pet1 and pet2:<\/p>\n
<\/p>\n
<\/p>\n
Doc 1: {\"pet1\": \"dog\", \"pet2\": \"dog\"}\nDoc 2: {\"pet1\": \"dog\", \"pet2\": \"cat\"}<\/span><\/pre>\n
<\/p>\n
<\/p>\n
We\u2019ll do a multi_match including those two fields like this:<\/p>\n
<\/p>\n
<\/p>\n
GET test\/_search\n{\n  \"query\": {\n   \"multi_match\": {\n    \"query\": \"dog cat\", \n    \"fields\": [\u201cpet1\u201d, \"pet2\"],\n    \"type\": \"most_fields\"\n  }\n }\n}<\/span><\/pre>\n
<\/p>\n
<\/p>\n
Since Document 2 matches both of our query terms we\u2019d certainly hope that it does better than Document 1. That\u2019s the lesson we took away from Query 4, right? Well, here are the results:<\/p>\n
<\/p>\n
<\/p>\n
\n\n\n\n\n\n
ID<\/strong><\/td>\nPet 1<\/strong><\/td>\nPet 2<\/strong><\/td>\nScore<\/strong><\/td>\n<\/tr>\n
1<\/td>\ndog<\/td>\ndog<\/td>\n0.87<\/td>\n<\/tr>\n
2<\/td>\ndog<\/td>\ncat<\/td>\n0.87<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
<\/p>\n
<\/p>\n
You can see the documents are tied. It might look like \u201ccat\u201d is a rare term that should help Document 2 rise to the top, but within the Pet 2 field, \u201ccat\u201d and \u201cdog\u201d have the\u00a0same<\/em>\u00a0document frequencies, and the scoring is done on a per-field basis. It also looks like Document 2 should get an advantage for matching\u00a0more<\/em>\u00a0of the query terms, but again, the scoring is done a per-field basis: when we compute the score in the Pet 1 field, both documents do the same; when we compute the score in the Pet 2 field, both documents again do the same.<\/p>\n
<\/p>\n
<\/p>\n
Does this contradict what we learned from Query 4? Not quite, but it warrants a refinement of the earlier lesson:<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
Matching more query terms within the same field is good. But there\u2019s no advantage when the matches happen across fields.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
If you\u2019re not happy with this situation, there are some things you can do. You can combine the contents of Pet 1 and Pet 2 into a single field. You can also switch from the\u00a0most_fields<\/strong>\u00a0to the\u00a0cross_fields<\/strong>\u00a0query type to simulate a single field. (Just be aware that cross_fields has some other consequences on scoring, changing the procedure from field-centric to term-centric. We won\u2019t go into details here.)<\/p>\n
<\/p>\n
<\/p>\n
Query 9: orange dog<\/h4>\n
<\/p>\n
<\/p>\n
Now let\u2019s revisit the lesson from Query 5, where we saw that matches in shorter fields are better. We\u2019re going to search for \u201corange dog.\u201d We have a \u201cdog\u201d document with a description mentioning that the dog is brown. And we have a \u201ccat\u201d document with a description mentioning that the cat is sometimes orange. Notice that the dog document has a longer description than the cat document, and both descriptions are longer than the contents of the type field.<\/p>\n
<\/p>\n
<\/p>\n
Doc 1: {\u201ctype\u201d: \u201cdog\u201d, \u201cdescription\u201d: \u201cA sweet and loving pet that is always eager to play. Brown coat. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non nibh sagittis, mollis ex a, scelerisque nisl. Ut vitae pellentesque magna, ut tristique nisi. Maecenas ut urna a elit posuere scelerisque. Suspendisse vel urna turpis. Mauris viverra fermentum ullamcorper. Duis ac lacus nibh. Nulla auctor lacus in purus vulputate, maximus ultricies augue scelerisque.\u201d}\n\nDoc 2: {\u201ctype\u201d: \u201ccat\u201d, \u201cdescription\u201d: \u201cPuzzlingly grumpy. Occasionally turns orange.\u201d}<\/span><\/pre>\n
<\/p>\n
<\/p>\n
We\u2019ll do a multi_match like this:<\/p>\n
<\/p>\n
<\/p>\n
GET test\/_search\n{\n  \"query\": {\n   \"multi_match\": {\n    \"query\": \"orange dog\", \n    \"fields\": [\"type\", \u201cdescription\"],\n    \"type\": \"most_fields\"\n  }\n }\n}<\/span><\/pre>\n
<\/p>\n
<\/p>\n
What\u2019s going to take precedence here: the match for \u201cdog\u201d in the type field, which is a really short field, or the match for \u201corange\u201d in the description field, which is significantly longer? If matches in shorter fields are better, shouldn\u2019t \u201cdog\u201d win here? In fact, the results look like this:<\/p>\n
<\/p>\n
<\/p>\n
\n\n\n\n\n\n
ID<\/td>\nType<\/strong><\/td>\nDescription<\/strong><\/td>\nScore<\/strong><\/td>\n<\/tr>\n
1<\/td>\ndog<\/td>\nA sweet\u2026 brown\u2026<\/td>\n0.69<\/td>\n<\/tr>\n
2<\/td>\ncat<\/td>\nPuzzlingly grumpy\u2026 orange.<\/td>\n1.06<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
<\/p>\n
<\/p>\n
The match for \u201corange\u201d in the Description field is sending Document 2 to the top even though that match occurs in a longer field body than the match for \u201cdog\u201d in Document 1. Does this contradict what we learned about matches in short and long fields from Query 5? No, but it points to something we hadn\u2019t mentioned. The lesson is that:<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
\u201cShortness\u201d is relative to the field\u2019s average.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
Within the type field, \u201cdog\u201d and \u201ccat\u201d don\u2019t get an advantage for being short because, in fact, they\u2019re both of average length for that field. On the other hand, the description for the cat is shorter than average for the description field overall, so it gets a benefit for being \u201cshort.\u201d<\/p>\n
<\/p>\n
<\/p>\n
Query 10: abcd efghijklmnopqrstuvwxyz<\/h4>\n
<\/p>\n
<\/p>\n
Here\u2019s an easy one to close out our set of examples. I\u2019ve divided the alphabet into two query terms. One of them is a short term, \u201cabcd,\u201d and the other is a long term, \u201cefghijklmnopqrstuvwxyz.\u201d I\u2019m going to search for both terms together: \u201cabcd efghijklmnopqrstuvwxyz.\u201d My index has one match for each term:<\/p>\n
<\/p>\n
<\/p>\n
Doc 1: \"abcd\"\nDoc 2: \"efghijklmnopqrstuvwxyz\"<\/span><\/pre>\n
<\/p>\n
<\/p>\n
Which document is going to do the best? The results look like this:<\/p>\n
<\/p>\n
<\/p>\n
\n\n\n\n\n\n
ID<\/td>\nTitle<\/strong><\/td>\nScore<\/strong><\/td>\n<\/tr>\n
1<\/td>\nabcd<\/td>\n0.69<\/td>\n<\/tr>\n
2<\/td>\nefghijklmnopqrstuvwxyz<\/td>\n0.69<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
<\/p>\n
<\/p>\n
Why are the documents tied if it\u2019s true that matches in shorter fields are better? The lesson is that<\/p>\n
<\/p>\n
<\/p>\n
When we talk about a short or long field, we\u2019re talking about how many\u00a0terms<\/em>\u00a0the field contains, not how many\u00a0characters<\/em>. In this example, the titles for both documents are of length 1.<\/p>\n
<\/p>\n
<\/p>\n
\n
\n
Term length is not significant.<\/p>\n<\/blockquote>\n<\/figure>\n
<\/p>\n
<\/p>\n
That\u2019s it for now. Hopefully these examples have helped build your intuitions about how scoring works in Elasticsearch and Solr. Thanks for following along! If you\u2019d like to go further and understand how the BM25 scoring function actually achieves the behaviors we\u2019ve seen here, check out our companion article on\u00a0Understanding TF-IDF and BM25<\/a>.<\/p>\n
<\/p>\t\t\t\t\t\t\t<\/div>\r\n\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div><\/div>\r\n\t\t<\/section>\r\n\t\t\t\t
\r\n\t\t\t\t\t\t
\r\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t
\n\t\t\t\tPrevious Post<\/span>Robotics, Search & AI<\/span><\/span><\/a>\t\t\t<\/div>\n\t\t\t\t\t\t
\n\t\t\t\tNext Post<\/span>Relevancy Tuning in Elastic<\/span><\/span><\/a>\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div><\/div>\r\n\t\t<\/section>\r\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"
Instead of delving into the mathematical definitions of TF-IDF and BM25, we will help you develop an intuitive understanding of these metrics using a series of simple examples.<\/p>\n","protected":false},"author":4,"featured_media":29724,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[38,43,36,37],"tags":[],"class_list":{"0":"post-26175","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-elasticsearch","8":"category-opensearch","9":"category-search","10":"category-solr"},"aioseo_notices":[],"post_meta_fields":{"_wp_page_template":["elementor_theme"],"_edit_lock":["1714530231:7"],"_edit_last":["7"],"_customize_sidebars":["yes"],"thegem_page_data":["a:190:{s:10:\"title_show\";s:7:\"default\";s:11:\"title_style\";s:1:\"2\";s:14:\"title_template\";s:5:\"27835\";s:23:\"title_use_page_settings\";i:0;s:12:\"title_xlarge\";i:0;s:18:\"title_rich_content\";i:0;s:13:\"title_content\";s:0:\"\";s:21:\"title_background_type\";s:5:\"color\";s:22:\"title_background_image\";s:0:\"\";s:29:\"title_background_image_repeat\";i:0;s:27:\"title_background_position_x\";s:6:\"center\";s:27:\"title_background_position_y\";s:3:\"top\";s:21:\"title_background_size\";s:5:\"cover\";s:28:\"title_background_image_color\";s:0:\"\";s:30:\"title_background_image_overlay\";s:0:\"\";s:30:\"title_background_gradient_type\";s:6:\"linear\";s:31:\"title_background_gradient_angle\";i:90;s:34:\"title_background_gradient_position\";s:13:\"center center\";s:38:\"title_background_gradient_point1_color\";s:9:\"#00BCD4BF\";s:41:\"title_background_gradient_point1_position\";i:0;s:38:\"title_background_gradient_point2_color\";s:9:\"#354093BF\";s:41:\"title_background_gradient_point2_position\";i:100;s:23:\"title_background_effect\";s:6:\"normal\";s:36:\"title_background_ken_burns_direction\";s:7:\"zoom_in\";s:43:\"title_background_ken_burns_transition_speed\";i:15000;s:37:\"title_background_video_play_on_mobile\";i:0;s:22:\"title_background_color\";s:7:\"#333144\";s:27:\"title_background_video_type\";s:0:\"\";s:22:\"title_background_video\";s:0:\"\";s:35:\"title_background_video_aspect_ratio\";s:0:\"\";s:36:\"title_background_video_overlay_color\";s:0:\"\";s:38:\"title_background_video_overlay_opacity\";s:0:\"\";s:29:\"title_background_video_poster\";s:0:\"\";s:19:\"title_menu_on_video\";s:0:\"\";s:16:\"title_text_color\";s:7:\"#ffffff\";s:24:\"title_excerpt_text_color\";s:7:\"#ffffff\";s:13:\"title_excerpt\";s:0:\"\";s:17:\"title_title_width\";i:0;s:19:\"title_excerpt_width\";i:0;s:22:\"title_font_preset_html\";s:0:\"\";s:23:\"title_font_preset_style\";s:0:\"\";s:24:\"title_font_preset_weight\";s:0:\"\";s:27:\"title_font_preset_transform\";s:0:\"\";s:30:\"title_excerpt_font_preset_html\";s:0:\"\";s:31:\"title_excerpt_font_preset_style\";s:0:\"\";s:32:\"title_excerpt_font_preset_weight\";s:0:\"\";s:35:\"title_excerpt_font_preset_transform\";s:0:\"\";s:17:\"title_padding_top\";i:80;s:24:\"title_padding_top_tablet\";i:80;s:24:\"title_padding_top_mobile\";i:80;s:20:\"title_padding_bottom\";i:80;s:27:\"title_padding_bottom_tablet\";i:80;s:27:\"title_padding_bottom_mobile\";i:80;s:18:\"title_padding_left\";i:0;s:25:\"title_padding_left_tablet\";i:0;s:25:\"title_padding_left_mobile\";i:0;s:19:\"title_padding_right\";i:0;s:26:\"title_padding_right_tablet\";i:0;s:26:\"title_padding_right_mobile\";i:0;s:16:\"title_top_margin\";s:0:\"\";s:23:\"title_top_margin_tablet\";i:0;s:23:\"title_top_margin_mobile\";i:0;s:24:\"title_excerpt_top_margin\";i:18;s:31:\"title_excerpt_top_margin_tablet\";i:18;s:31:\"title_excerpt_top_margin_mobile\";i:18;s:17:\"title_breadcrumbs\";i:1;s:15:\"title_alignment\";s:0:\"\";s:15:\"title_icon_pack\";s:7:\"elegant\";s:10:\"title_icon\";s:0:\"\";s:16:\"title_icon_color\";s:0:\"\";s:18:\"title_icon_color_2\";s:0:\"\";s:27:\"title_icon_background_color\";s:0:\"\";s:16:\"title_icon_shape\";s:6:\"circle\";s:23:\"title_icon_border_color\";s:0:\"\";s:15:\"title_icon_size\";s:5:\"large\";s:16:\"title_icon_style\";s:0:\"\";s:18:\"title_icon_opacity\";d:0;s:25:\"breadcrumbs_default_color\";s:0:\"\";s:24:\"breadcrumbs_active_color\";s:0:\"\";s:23:\"breadcrumbs_hover_color\";s:0:\"\";s:27:\"title_breadcrumbs_alignment\";s:6:\"center\";s:18:\"header_transparent\";i:0;s:14:\"header_opacity\";i:0;s:22:\"header_menu_logo_light\";i:0;s:20:\"header_hide_top_area\";s:7:\"default\";s:27:\"header_hide_top_area_tablet\";s:7:\"default\";s:27:\"header_hide_top_area_mobile\";s:7:\"default\";s:9:\"menu_show\";s:7:\"default\";s:12:\"menu_options\";s:7:\"default\";s:18:\"header_custom_menu\";i:0;s:27:\"header_top_area_transparent\";i:0;s:23:\"header_top_area_opacity\";i:0;s:16:\"top_area_options\";s:7:\"default\";s:13:\"header_source\";s:7:\"default\";s:14:\"header_builder\";s:1:\"0\";s:29:\"header_builder_sticky_desktop\";i:0;s:28:\"header_builder_sticky_mobile\";i:0;s:34:\"header_builder_sticky_hide_desktop\";i:0;s:33:\"header_builder_sticky_hide_mobile\";i:1;s:21:\"header_builder_sticky\";s:1:\"0\";s:29:\"header_builder_sticky_opacity\";i:80;s:26:\"header_builder_light_color\";s:7:\"#FFFFFF\";s:32:\"header_builder_light_color_hover\";s:7:\"#00bcd4\";s:20:\"main_background_type\";b:0;s:21:\"main_background_color\";s:7:\"#ffffff\";s:21:\"main_background_image\";s:0:\"\";s:28:\"main_background_image_repeat\";i:0;s:26:\"main_background_position_x\";s:6:\"center\";s:26:\"main_background_position_y\";s:6:\"center\";s:20:\"main_background_size\";s:4:\"auto\";s:27:\"main_background_image_color\";s:0:\"\";s:29:\"main_background_image_overlay\";s:0:\"\";s:29:\"main_background_gradient_type\";s:6:\"linear\";s:30:\"main_background_gradient_angle\";i:90;s:33:\"main_background_gradient_position\";s:0:\"\";s:37:\"main_background_gradient_point1_color\";s:9:\"#E9ECDAFF\";s:40:\"main_background_gradient_point1_position\";i:0;s:37:\"main_background_gradient_point2_color\";s:9:\"#D5F6FAFF\";s:40:\"main_background_gradient_point2_position\";i:100;s:23:\"main_background_pattern\";s:0:\"\";s:19:\"content_padding_top\";i:135;s:26:\"content_padding_top_tablet\";s:0:\"\";s:26:\"content_padding_top_mobile\";s:0:\"\";s:22:\"content_padding_bottom\";i:110;s:29:\"content_padding_bottom_tablet\";s:0:\"\";s:29:\"content_padding_bottom_mobile\";s:0:\"\";s:20:\"content_area_options\";s:7:\"default\";s:18:\"footer_custom_show\";s:7:\"default\";s:13:\"footer_custom\";s:5:\"24822\";s:19:\"footer_hide_default\";s:7:\"default\";s:23:\"footer_hide_widget_area\";s:7:\"default\";s:16:\"effects_disabled\";i:0;s:17:\"effects_one_pager\";i:0;s:23:\"effects_parallax_footer\";i:0;s:24:\"effects_no_bottom_margin\";i:0;s:21:\"effects_no_top_margin\";i:0;s:19:\"redirect_to_subpage\";i:0;s:19:\"effects_hide_header\";s:7:\"default\";s:19:\"effects_hide_footer\";s:7:\"default\";s:21:\"effects_page_scroller\";i:0;s:28:\"effects_page_scroller_mobile\";i:0;s:26:\"effects_page_scroller_type\";s:8:\"advanced\";s:22:\"fullpage_disabled_dots\";i:0;s:19:\"fullpage_style_dots\";s:7:\"outline\";s:31:\"fullpage_disabled_tooltips_dots\";i:0;s:25:\"fullpage_fixed_background\";b:0;s:26:\"fullpage_enable_continuous\";i:0;s:24:\"fullpage_disabled_mobile\";i:0;s:22:\"fullpage_scroll_effect\";s:6:\"normal\";s:21:\"enable_page_preloader\";s:7:\"default\";s:14:\"slideshow_type\";s:0:\"\";s:19:\"slideshow_slideshow\";s:0:\"\";s:21:\"slideshow_layerslider\";s:0:\"\";s:19:\"slideshow_revslider\";s:0:\"\";s:19:\"slideshow_preloader\";i:1;s:12:\"sidebar_show\";s:8:\"disabled\";s:16:\"sidebar_position\";s:4:\"left\";s:14:\"sidebar_sticky\";i:0;s:24:\"product_header_separator\";i:0;s:23:\"page_layout_breadcrumbs\";s:7:\"default\";s:37:\"page_layout_breadcrumbs_default_color\";s:9:\"#99A9B5FF\";s:36:\"page_layout_breadcrumbs_active_color\";s:9:\"#3C3950FF\";s:35:\"page_layout_breadcrumbs_hover_color\";s:9:\"#3C3950FF\";s:33:\"page_layout_breadcrumbs_alignment\";s:4:\"left\";s:38:\"page_layout_breadcrumbs_bottom_spacing\";s:1:\"0\";s:37:\"page_layout_breadcrumbs_shop_category\";i:0;s:18:\"delay_js_execution\";i:0;s:13:\"disable_cache\";i:0;s:27:\"title_video_overlay_opacity\";s:0:\"\";s:31:\"title_breadcrumbs_shop_category\";s:1:\"0\";s:20:\"title_padding_locked\";s:0:\"\";s:27:\"title_padding_tablet_locked\";s:0:\"\";s:27:\"title_padding_mobile_locked\";s:0:\"\";s:24:\"title_background_pattern\";s:0:\"\";s:30:\"title_background_video_overlay\";s:0:\"\";s:16:\"title_icon__pack\";s:0:\"\";s:21:\"title_icon_shape_show\";s:0:\"\";s:25:\"footer_widget_woocommerce\";s:1:\"1\";s:19:\"portfolio_item_data\";a:9:{s:8:\"back_url\";s:0:\"\";s:9:\"highlight\";s:0:\"\";s:14:\"highlight_type\";s:0:\"\";s:14:\"overview_title\";s:0:\"\";s:16:\"overview_summary\";s:0:\"\";s:12:\"project_link\";s:0:\"\";s:12:\"project_text\";s:0:\"\";s:9:\"fullwidth\";s:0:\"\";s:19:\"project_button_show\";s:0:\"\";}s:23:\"portfolio_elements_data\";a:7:{s:23:\"portfolio_page_elements\";s:7:\"default\";s:19:\"portfolio_hide_date\";s:0:\"\";s:19:\"portfolio_hide_sets\";s:0:\"\";s:20:\"portfolio_hide_likes\";s:0:\"\";s:22:\"portfolio_hide_socials\";s:0:\"\";s:29:\"portfolio_hide_top_navigation\";s:0:\"\";s:32:\"portfolio_hide_bottom_navigation\";s:0:\"\";}s:17:\"product_item_data\";a:109:{s:9:\"highlight\";s:0:\"\";s:14:\"highlight_type\";s:7:\"squared\";s:28:\"thegem_product_disable_hover\";s:1:\"0\";s:10:\"size_guide\";s:7:\"default\";s:16:\"size_guide_image\";s:0:\"\";s:23:\"product_layout_settings\";s:7:\"default\";s:21:\"product_layout_source\";s:7:\"default\";s:24:\"product_builder_template\";s:0:\"\";s:19:\"product_page_layout\";s:7:\"default\";s:25:\"product_page_layout_style\";s:15:\"horizontal_tabs\";s:28:\"product_page_layout_centered\";s:1:\"0\";s:39:\"product_page_layout_centered_top_margin\";s:2:\"42\";s:34:\"product_page_layout_centered_boxed\";s:1:\"0\";s:40:\"product_page_layout_centered_boxed_color\";s:0:\"\";s:29:\"product_page_layout_fullwidth\";s:1:\"0\";s:26:\"product_page_layout_sticky\";s:1:\"0\";s:33:\"product_page_layout_sticky_offset\";s:1:\"0\";s:28:\"product_page_skeleton_loader\";s:1:\"0\";s:30:\"product_page_layout_background\";s:0:\"\";s:30:\"product_page_layout_title_area\";s:8:\"disabled\";s:29:\"product_page_ajax_add_to_cart\";s:1:\"1\";s:31:\"product_page_desc_review_source\";s:17:\"extra_description\";s:31:\"product_page_desc_review_layout\";s:4:\"tabs\";s:42:\"product_page_desc_review_layout_tabs_style\";s:10:\"horizontal\";s:46:\"product_page_desc_review_layout_tabs_alignment\";s:4:\"left\";s:44:\"product_page_desc_review_layout_acc_position\";s:13:\"below_gallery\";s:65:\"product_page_desc_review_layout_one_by_one_description_background\";s:9:\"#F4F6F7FF\";s:69:\"product_page_desc_review_layout_one_by_one_additional_info_background\";s:9:\"#FFFFFFFF\";s:61:\"product_page_desc_review_layout_one_by_one_reviews_background\";s:9:\"#F4F6F7FF\";s:36:\"product_page_desc_review_description\";s:1:\"1\";s:42:\"product_page_desc_review_description_title\";s:11:\"Description\";s:40:\"product_page_desc_review_additional_info\";s:1:\"1\";s:46:\"product_page_desc_review_additional_info_title\";s:15:\"Additional Info\";s:32:\"product_page_desc_review_reviews\";s:1:\"1\";s:38:\"product_page_desc_review_reviews_title\";s:7:\"Reviews\";s:36:\"product_page_button_add_to_cart_text\";s:11:\"Add to Cart\";s:36:\"product_page_button_add_to_cart_icon\";s:4:\"f1e7\";s:41:\"product_page_button_add_to_cart_icon_pack\";s:8:\"material\";s:45:\"product_page_button_add_to_cart_icon_position\";s:4:\"left\";s:40:\"product_page_button_add_to_wishlist_icon\";s:4:\"f37b\";s:45:\"product_page_button_add_to_wishlist_icon_pack\";s:8:\"material\";s:42:\"product_page_button_added_to_wishlist_icon\";s:4:\"f377\";s:47:\"product_page_button_added_to_wishlist_icon_pack\";s:8:\"material\";s:41:\"product_page_button_clear_attributes_text\";s:15:\"Clear selection\";s:31:\"product_page_elements_prev_next\";s:1:\"1\";s:38:\"product_page_elements_preview_on_hover\";s:1:\"1\";s:34:\"product_page_elements_back_to_shop\";s:1:\"1\";s:39:\"product_page_elements_back_to_shop_link\";s:9:\"main_shop\";s:50:\"product_page_elements_back_to_shop_link_custom_url\";s:0:\"\";s:27:\"product_page_elements_title\";s:1:\"1\";s:32:\"product_page_elements_attributes\";s:1:\"0\";s:37:\"product_page_elements_attributes_data\";s:0:\"\";s:29:\"product_page_elements_reviews\";s:1:\"1\";s:34:\"product_page_elements_reviews_text\";s:16:\"customer reviews\";s:27:\"product_page_elements_price\";s:1:\"1\";s:41:\"product_page_elements_price_strikethrough\";s:1:\"1\";s:33:\"product_page_elements_description\";s:1:\"1\";s:34:\"product_page_elements_stock_amount\";s:1:\"1\";s:39:\"product_page_elements_stock_amount_text\";s:17:\"Products in stock\";s:32:\"product_page_elements_size_guide\";s:1:\"1\";s:25:\"product_page_elements_sku\";s:1:\"1\";s:31:\"product_page_elements_sku_title\";s:3:\"SKU\";s:32:\"product_page_elements_categories\";s:1:\"1\";s:38:\"product_page_elements_categories_title\";s:10:\"Categories\";s:26:\"product_page_elements_tags\";s:1:\"1\";s:32:\"product_page_elements_tags_title\";s:4:\"Tags\";s:27:\"product_page_elements_share\";s:1:\"1\";s:33:\"product_page_elements_share_title\";s:5:\"Share\";s:36:\"product_page_elements_share_facebook\";s:1:\"1\";s:35:\"product_page_elements_share_twitter\";s:1:\"1\";s:37:\"product_page_elements_share_pinterest\";s:1:\"1\";s:34:\"product_page_elements_share_tumblr\";s:1:\"1\";s:36:\"product_page_elements_share_linkedin\";s:1:\"1\";s:34:\"product_page_elements_share_reddit\";s:1:\"1\";s:28:\"product_page_elements_upsell\";s:1:\"1\";s:34:\"product_page_elements_upsell_title\";s:17:\"You may also like\";s:44:\"product_page_elements_upsell_title_alignment\";s:4:\"left\";s:34:\"product_page_elements_upsell_items\";s:2:\"-1\";s:44:\"product_page_elements_upsell_columns_desktop\";s:2:\"4x\";s:43:\"product_page_elements_upsell_columns_tablet\";s:2:\"3x\";s:43:\"product_page_elements_upsell_columns_mobile\";s:2:\"2x\";s:40:\"product_page_elements_upsell_columns_100\";s:1:\"5\";s:29:\"product_page_elements_related\";s:1:\"1\";s:35:\"product_page_elements_related_title\";s:16:\"Related Products\";s:45:\"product_page_elements_related_title_alignment\";s:4:\"left\";s:35:\"product_page_elements_related_items\";s:2:\"-1\";s:45:\"product_page_elements_related_columns_desktop\";s:2:\"4x\";s:44:\"product_page_elements_related_columns_tablet\";s:2:\"3x\";s:44:\"product_page_elements_related_columns_mobile\";s:2:\"2x\";s:41:\"product_page_elements_related_columns_100\";s:1:\"5\";s:15:\"product_gallery\";s:7:\"enabled\";s:20:\"product_gallery_type\";s:10:\"horizontal\";s:31:\"product_gallery_column_position\";s:4:\"left\";s:28:\"product_gallery_column_width\";s:2:\"50\";s:26:\"product_gallery_show_image\";s:5:\"hover\";s:20:\"product_gallery_zoom\";s:1:\"1\";s:24:\"product_gallery_lightbox\";s:1:\"1\";s:22:\"product_gallery_labels\";s:1:\"1\";s:26:\"product_gallery_label_sale\";s:1:\"1\";s:25:\"product_gallery_label_new\";s:1:\"1\";s:31:\"product_gallery_label_out_stock\";s:1:\"1\";s:27:\"product_gallery_auto_height\";s:1:\"1\";s:30:\"product_gallery_elements_color\";s:0:\"\";s:28:\"product_gallery_grid_columns\";s:2:\"1x\";s:25:\"product_gallery_grid_gaps\";s:2:\"42\";s:30:\"product_gallery_grid_gaps_hide\";s:1:\"0\";s:31:\"product_gallery_grid_top_margin\";s:1:\"0\";s:30:\"product_gallery_video_autoplay\";s:1:\"0\";s:15:\"size_guide_text\";s:10:\"Size guide\";}s:25:\"product_archive_item_data\";a:2:{s:29:\"product_archive_layout_source\";s:7:\"default\";s:32:\"product_archive_builder_template\";s:0:\"\";}s:22:\"blog_archive_item_data\";a:2:{s:26:\"blog_archive_layout_source\";s:7:\"default\";s:29:\"blog_archive_builder_template\";s:0:\"\";}s:24:\"options_current_contents\";N;s:16:\"options_modified\";N;s:34:\"options_outside_parameter_modified\";b:0;s:22:\"options_saved_contents\";N;s:8:\"settings\";a:3:{s:5:\"theme\";s:5:\"light\";s:24:\"background_image_gallery\";a:0:{}s:21:\"colorpicker_favorites\";a:1:{s:7:\"default\";a:0:{}}}s:26:\"delay_js_execution_desktop\";s:1:\"0\";s:25:\"delay_js_execution_mobile\";s:1:\"0\";}"],"thegem_post_general_item_data":["a:22:{s:20:\"post_layout_settings\";s:7:\"default\";s:18:\"post_layout_source\";s:7:\"default\";s:21:\"post_builder_template\";s:1:\"0\";s:26:\"show_featured_posts_slider\";i:0;s:21:\"show_featured_content\";s:8:\"disabled\";s:10:\"video_type\";s:7:\"youtube\";s:5:\"video\";s:0:\"\";s:18:\"video_aspect_ratio\";s:0:\"\";s:10:\"quote_text\";s:0:\"\";s:12:\"quote_author\";s:0:\"\";s:16:\"quote_background\";s:0:\"\";s:18:\"quote_author_color\";s:0:\"\";s:5:\"audio\";s:0:\"\";s:7:\"gallery\";i:0;s:18:\"gallery_autoscroll\";i:0;s:9:\"highlight\";i:0;s:14:\"highlight_type\";s:7:\"squared\";s:15:\"highlight_style\";s:7:\"default\";s:31:\"highlight_title_left_background\";s:0:\"\";s:26:\"highlight_title_left_color\";s:0:\"\";s:32:\"highlight_title_right_background\";s:0:\"\";s:27:\"highlight_title_right_color\";s:0:\"\";}"],"thegem_show_featured_posts_slider":["0"],"_zilla_likes":["0"],"_wp_old_date":["2021-03-10"],"_thumbnail_id":["29724"],"thegem_page_data_old":["a:46:{s:11:\"title_style\";s:1:\"1\";s:14:\"title_template\";i:0;s:23:\"title_use_page_settings\";i:0;s:12:\"title_xlarge\";i:0;s:18:\"title_rich_content\";i:0;s:13:\"title_content\";s:0:\"\";s:22:\"title_background_image\";s:0:\"\";s:25:\"title_background_parallax\";i:0;s:22:\"title_background_color\";s:0:\"\";s:16:\"title_video_type\";s:0:\"\";s:22:\"title_video_background\";s:0:\"\";s:24:\"title_video_aspect_ratio\";s:0:\"\";s:25:\"title_video_overlay_color\";s:0:\"\";s:27:\"title_video_overlay_opacity\";s:0:\"\";s:18:\"title_video_poster\";s:0:\"\";s:19:\"title_menu_on_video\";s:0:\"\";s:16:\"title_text_color\";s:0:\"\";s:24:\"title_excerpt_text_color\";s:0:\"\";s:13:\"title_excerpt\";s:0:\"\";s:17:\"title_title_width\";s:1:\"0\";s:19:\"title_excerpt_width\";s:1:\"0\";s:17:\"title_padding_top\";s:2:\"80\";s:20:\"title_padding_bottom\";s:2:\"80\";s:16:\"title_top_margin\";s:1:\"0\";s:24:\"title_excerpt_top_margin\";s:2:\"18\";s:17:\"title_breadcrumbs\";s:1:\"1\";s:15:\"title_alignment\";s:0:\"\";s:15:\"title_icon_pack\";s:7:\"elegant\";s:10:\"title_icon\";s:0:\"\";s:16:\"title_icon_color\";s:0:\"\";s:18:\"title_icon_color_2\";s:0:\"\";s:27:\"title_icon_background_color\";s:0:\"\";s:16:\"title_icon_shape\";s:6:\"circle\";s:23:\"title_icon_border_color\";s:0:\"\";s:15:\"title_icon_size\";s:5:\"large\";s:16:\"title_icon_style\";s:0:\"\";s:18:\"title_icon_opacity\";d:0;s:14:\"header_opacity\";s:1:\"0\";s:23:\"header_top_area_opacity\";s:1:\"0\";s:13:\"footer_custom\";s:5:\"24822\";s:16:\"sidebar_position\";s:0:\"\";s:14:\"slideshow_type\";s:0:\"\";s:19:\"slideshow_slideshow\";s:0:\"\";s:19:\"fullpage_style_dots\";s:7:\"outline\";s:22:\"fullpage_scroll_effect\";s:6:\"normal\";s:14:\"sidebar_sticky\";i:0;}"],"thegem_post_general_item_data_old":["a:19:{s:26:\"show_featured_posts_slider\";i:0;s:21:\"show_featured_content\";i:0;s:10:\"video_type\";s:7:\"youtube\";s:5:\"video\";s:0:\"\";s:18:\"video_aspect_ratio\";s:0:\"\";s:10:\"quote_text\";s:0:\"\";s:12:\"quote_author\";s:0:\"\";s:16:\"quote_background\";s:0:\"\";s:18:\"quote_author_color\";s:0:\"\";s:5:\"audio\";s:0:\"\";s:7:\"gallery\";i:0;s:18:\"gallery_autoscroll\";i:0;s:9:\"highlight\";i:0;s:14:\"highlight_type\";s:7:\"squared\";s:15:\"highlight_style\";s:7:\"default\";s:31:\"highlight_title_left_background\";s:0:\"\";s:26:\"highlight_title_left_color\";s:0:\"\";s:32:\"highlight_title_right_background\";s:0:\"\";s:27:\"highlight_title_right_color\";s:0:\"\";}"],"_elementor_edit_mode":["builder"],"_elementor_template_type":["wp-post"],"_elementor_version":["3.18.3"],"_elementor_data":["[{\"id\":\"621a0122\",\"elType\":\"section\",\"settings\":[],\"elements\":[{\"id\":\"692d9fe3\",\"elType\":\"column\",\"settings\":{\"_column_size\":100,\"thegem_column_breakpoints_list\":[]},\"elements\":[{\"id\":\"22d2aadc\",\"elType\":\"widget\",\"settings\":{\"editor\":\"
<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
The built-in scoring mechanism in Elasticsearch and Solr can seem mysterious to beginners and experienced practitioners alike. Instead of delving into the mathematical definitions of TF-IDF and BM25, this article will help you develop an intuitive understanding of these metrics by walking you through a series of simple examples. Each example consists of a query and list of several indexed documents. As you read along, try to guess which document comes up on top for each query. In each case, we will examine why that particular document gets the highest score and we\\u2019ll extract the general principle behind this behavior. A set of six examples will be followed by an extra credit section focusing on more advanced topics. Along with illustrating all of the key behaviors of BM25, our examples will touch on some of the gotchas around scoring in cluster scenario, where shards and replicas come into play. This article aims to teach you, in a short time and without any math, everything you\\u2019ll ever need to know about scoring. Having a solid understanding of scoring will prepare you to better diagnose relevance problems and improve relevance in real-world applications.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Query 1: dog<\\\/h4>\\n
<\\\/p>\\n
<\\\/p>\\n
Let\\u2019s say I search for \\u201cdog\\u201d and there are only three documents in my index, as shown below. Which one of these documents is going to come up on top?<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 1: \\\"dog\\\"\\n\\nDoc 2: \\\"dog dog\\\"\\n\\nDoc 3: \\\"dog dog dog<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
If you\\u2019re not quite sure, that\\u2019s good, because I haven\\u2019t given you enough of the context to know the answer. All the queries in this article were tested in Elasticsearch 7.4 where BM25 is the default scoring algorithm and its parameters are set as k=1.2 and b=0.7. (Please ignore that if it\\u2019s meaningless to you.) In most of the examples, we\\u2019ll assume the documents have a single text field called \\u201ctitle\\u201d that uses the standard analyzer:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\\"mappings\\\": {\\n    \\\"properties\\\": {\\n      \\\"title\\\": {\\n        \\\"type\\\": \\\"text\\\",\\n        \\\"analyzer\\\": \\\"standard\\\"\\n      }\\n    }\\n  }<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
For the most part, we\\u2019ll be doing simple\\u00a0match<\\\/strong>\\u00a0queries against the title field. Recall that the default boolean operator in Elasticsearch is\\u00a0OR<\\\/strong>. Here\\u2019s what our \\u201cdog\\u201d query looks like:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
GET test\\\/_search\\n{\\n  \\\"query\\\": {\\n      \\\"match\\\": {\\n          \\\"title\\\": \\\"dog\\\"\\n      } \\n  }\\n}<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
With those details out of the way, are you ready to tell me which one of those three documents (\\u201cdog,\\u201d \\u201cdog dog,\\u201d and \\u201cdog dog dog\\u201d) is going to get the highest score?<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Here are the results:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n\\n\\n\\n\\n\\n\\n
ID<\\\/strong><\\\/td>\\nTitle<\\\/strong><\\\/td>\\nScore<\\\/strong><\\\/td>\\n<\\\/tr>\\n
1<\\\/td>\\ndog<\\\/td>\\n0.167<\\\/td>\\n<\\\/tr>\\n
2<\\\/td>\\ndog dog<\\\/td>\\n0.183<\\\/td>\\n<\\\/tr>\\n
3<\\\/td>\\ndog dog dog<\\\/td>\\n0.189<\\\/strong><\\\/td>\\n<\\\/tr>\\n<\\\/tbody>\\n<\\\/table>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 3 gets the highest score because it has contains the highest number of tokens that match the query term. Another way to say this is that Doc 3 has the highest\\u00a0term frequency<\\\/strong>\\u00a0for the term in question. Of course if we were using the keyword analyzer, only Doc 1 would have matched the query, but we\\u2019re using the standard analyzer which breaks the title into multiple tokens. The moral of this story is that, from a scoring perspective at least,<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
High term frequency is good.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Before we move on, take a moment to compare Doc 1 and Doc 3. Notice that although Doc 3 has three times the term frequency for \\u201cdog\\u201d as Doc 1, its score isn\\u2019t three times as high. So while higher term frequency gives a higher score, its impact is not multiplicative.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Query 2: dog dog cat<\\\/h4>\\n
<\\\/p>\\n
<\\\/p>\\n
Now I\\u2019m searching for \\u201cdog dog cat\\u201d and there are only two documents in my index:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 1: \\\"cat\\\"\\nDoc 2: \\\"dog\\\"<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
Which one of these is going to come up on top? Or are they going to be tied?<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
In fact, \\u201cdog\\u201d is the winner here:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n\\n\\n\\n\\n\\n
ID<\\\/strong><\\\/td>\\nTitle<\\\/strong><\\\/td>\\nScore<\\\/strong><\\\/td>\\n<\\\/tr>\\n
1<\\\/td>\\ncat<\\\/td>\\n0.6<\\\/td>\\n<\\\/tr>\\n
2<\\\/td>\\ndog<\\\/td>\\n1.3<\\\/strong><\\\/td>\\n<\\\/tr>\\n<\\\/tbody>\\n<\\\/table>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Why does \\u201cdog\\u201d get twice the score of \\u201ccat?\\u201d The lesson here is that<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
Scores for each query term are summed.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Our query has two instances of \\u201cdog.\\u201d The score for the whole query is the sum of the scores for each term. Each instance of \\u201cdog\\u201d in our query is going to match the \\u201cdog\\u201d in Doc 2 and contribute roughly 0.6 to the score, for a total of 1.3. Using the standard analyzer, query terms aren\\u2019t deduplicated, so each instance of \\u201cdog\\u201d is treated separately. Doc 1 doesn\\u2019t get a similar advantage because our query only contains one instance of \\u201ccat.\\u201d<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Query 3: dog dog cat<\\\/h4>\\n
<\\\/p>\\n
<\\\/p>\\n
Now I\\u2019m executing the same query as before, \\u201cdog dog cat,\\u201d but my index is different. This time I have lots of \\u201cdog\\u201d documents:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 1: \\\"dog\\\"\\nDoc 2: \\\"dog\\\"\\nDoc 3: \\\"dog\\\"\\nDoc 4: \\\"dog\\\"\\nDoc 5: \\\"dog\\\"\\nDoc 6: \\\"dog\\\"\\nDoc 7: \\\"cat\\\"<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
What\\u2019s going to happen now? Do the \\u201cdog\\u201d documents still win over \\u201ccat\\u201d because my query mentions \\u201cdog\\u201d twice? Here are the results:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n
ID<\\\/strong><\\\/td>\\nTitle<\\\/strong><\\\/td>\\nScore<\\\/strong><\\\/td>\\n<\\\/tr>\\n
1<\\\/td>\\ndog<\\\/td>\\n0.4<\\\/td>\\n<\\\/tr>\\n
2<\\\/td>\\ndog<\\\/td>\\n0.4<\\\/td>\\n<\\\/tr>\\n
3<\\\/td>\\ndog<\\\/td>\\n0.4<\\\/td>\\n<\\\/tr>\\n
4<\\\/td>\\ndog<\\\/td>\\n0.4<\\\/td>\\n<\\\/tr>\\n
5<\\\/td>\\ndog<\\\/td>\\n0.4<\\\/td>\\n<\\\/tr>\\n
6<\\\/td>\\ndog<\\\/td>\\n0.4<\\\/td>\\n<\\\/tr>\\n
7<\\\/td>\\ncat<\\\/td>\\n1.5<\\\/strong><\\\/td>\\n<\\\/tr>\\n<\\\/tbody>\\n<\\\/table>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
The results are different this time because the terms have different\\u00a0document frequencies<\\\/strong>\\u00a0than before. A term\\u2019s document frequency is the number of documents in the index that contain the term. From a scoring perspective, low document frequency is good and high document frequency is bad. In this example, \\u201ccat\\u201d is a rare term in the index \\u2014 it has low document frequency \\u2014 so matches on that term help the score more than matches on \\u201cdog,\\u201d which is a common term. The lesson here is that:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
Matches for rarer terms are better.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
If I want to tell the search engine that a common term is particularly important to me in a certain scenario, I can\\u00a0boost<\\\/strong>\\u00a0the term. If I had executed my query with a boost of 7 on \\u201cdog,\\u201d the dog documents would come up above the cat document. Here\\u2019s how I\\u2019d set that up:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
GET test\\\/_search\\n{\\n \\\"query\\\": {\\n  \\\"query_string\\\": {\\n   \\\"query\\\": \\\"dog^7 cat\\\",\\n   \\\"fields\\\": [\\\"title\\\"]\\n  } \\n }\\n}<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
Query 4: dog cat<\\\/h4>\\n
<\\\/p>\\n
<\\\/p>\\n
In this example I\\u2019m searching for \\u201cdog cat,\\u201d and I\\u2019ve got three documents in my index: one with a lot of dogs, one with a lot of cats, and one with a single instance of dog and cat each, plus a lot of mats.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 1: \\\"dog dog dog dog dog dog dog\\\"\\nDoc 2: \\\"cat cat cat cat cat cat cat\\\"\\nDoc 3: \\\"dog cat mat mat mat mat mat\\\"<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
Which document comes up on top this time? Notice that in Doc 1 and Doc 2, every single term matches one of the query terms, whereas in Doc 3 there are five terms that don\\u2019t match anything. So the results might be a little surprising:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n\\n\\n\\n\\n\\n\\n
ID<\\\/td>\\nTitle<\\\/td>\\nScore<\\\/td>\\n<\\\/tr>\\n
1<\\\/td>\\ndog dog dog dog dog dog dog<\\\/td>\\n0.88<\\\/td>\\n<\\\/tr>\\n
2<\\\/td>\\ncat cat cat cat cat cat cat<\\\/td>\\n0.88<\\\/td>\\n<\\\/tr>\\n
3<\\\/td>\\ndog cat mat mat mat mat mat<\\\/td>\\n0.94<\\\/strong><\\\/td>\\n<\\\/tr>\\n<\\\/tbody>\\n<\\\/table>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Document 3 gets the highest score because it has matches for\\u00a0both<\\\/em>\\u00a0of the query terms, \\u201cdog\\u201d and \\u201ccat.\\u201d While Documents 1 and 2 have higher term frequency for \\u201cdog\\u201d and \\u201ccat\\u201d respectively, they each contain only one of the terms. The lesson is that<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
Matching more query terms is good.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Query 5: dog<\\\/h4>\\n
<\\\/p>\\n
<\\\/p>\\n
Now I\\u2019ll search for \\u201cdog\\u201d and there are only two documents in my index:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 1: \\\"dog cat zebra\\\"\\nDoc 2: \\\"dog cat<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
Both of these documents match my query, and both have some terms that don\\u2019t match. Which document does the best? Here are there results:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n\\n\\n\\n\\n\\n
ID<\\\/td>\\nTitle<\\\/td>\\nScore<\\\/td>\\n<\\\/tr>\\n
1<\\\/td>\\ndog cat zebra<\\\/td>\\n0.16<\\\/td>\\n<\\\/tr>\\n
2<\\\/td>\\ndog cat<\\\/td>\\n0.19<\\\/strong><\\\/td>\\n<\\\/tr>\\n<\\\/tbody>\\n<\\\/table>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
In this case, Document 2 does better because it is shorter. The thinking is that when a term occurs in a shorter document, we can be more confident that the term is significant to the document (or that the document is\\u00a0about<\\\/em>\\u00a0the term). When a term occurs in a longer document, we have less confidence that this occurrence is meaningful. The lesson here is that<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
Matches in shorter documents are better.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Query 6: orange dog<\\\/h4>\\n
<\\\/p>\\n
<\\\/p>\\n
Now let\\u2019s consider a scenario that\\u2019s a little more complicated than the previous ones. For the first time, our documents will have two fields, \\u201ccolor\\u201d and \\u201ctype.\\u201d<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 1: {\\\"color\\\": \\\"brown\\\", \\\"type\\\": \\\"dog\\\"}\\nDoc 2: {\\\"color\\\": \\\"brown\\\", \\\"type\\\": \\\"dog\\\"}\\nDoc 3: {\\\"color\\\": \\\"brown\\\", \\\"type\\\": \\\"cat\\\"}\\nDoc 4: {\\\"color\\\": \\\"orange\\\", \\\"type\\\": \\\"cat\\\"}<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
We\\u2019re searching for \\u201corange dog\\u201d but as you can see, there are no orange dogs in the index. There are two brown dogs, a brown cat, and an orange cat. Which one is going to come up on top?<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
I should mention that we\\u2019re searching across both fields using a\\u00a0multi_match<\\\/strong>\\u00a0like this:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
GET test\\\/_search\\n{\\n  \\\"query\\\": {\\n   \\\"multi_match\\\": {\\n    \\\"query\\\": \\\"orange dog\\\", \\n    \\\"fields\\\": [\\\"type\\\", \\\"color\\\"],\\n    \\\"type\\\": \\\"most_fields\\\"\\n  }\\n }\\n}<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
Here are the results:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n\\n\\n\\n\\n\\n\\n\\n
ID<\\\/td>\\nColor<\\\/td>\\nType<\\\/td>\\nScore<\\\/td>\\n<\\\/tr>\\n
1<\\\/td>\\nbrown<\\\/td>\\ndog<\\\/td>\\n0.6<\\\/td>\\n<\\\/tr>\\n
2<\\\/td>\\nbrown<\\\/td>\\ndog<\\\/td>\\n0.6<\\\/td>\\n<\\\/tr>\\n
3<\\\/td>\\nbrown<\\\/td>\\ncat<\\\/td>\\nN\\\/A<\\\/td>\\n<\\\/tr>\\n
4<\\\/td>\\norange<\\\/td>\\ncat<\\\/td>\\n1.2<\\\/td>\\n<\\\/tr>\\n<\\\/tbody>\\n<\\\/table>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
This example hints at some of the unexpected behavior that can arise when we search across multiple fields. The search engine doesn\\u2019t know which field is most important to us. If someone is searching for an orange dog, we might guess they\\u2019re more interested in seeing dogs than seeing arbitrary things that happen to be orange. (\\u201cOrange dog\\u201d would be very strange query to enter if you meant \\u201cshow me anything that\\u2019s orange.\\u201d) In this case, however, the color field is taking priority because \\u201corange\\u201d is a rare term within that field (there are 3 browns and only 1 orange). Within the type field, \\u201cdog\\u201d and \\u201ccat\\u201d have the same frequency. The orange cat comes up on top because a match for the rare term \\u201corange\\u201d is treated as more valuable than a match for \\u201cdog.\\u201d<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
If we want to give more weight to the \\u201ctype\\u201d field we can boost it like this:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
GET test\\\/_search\\n{\\n  \\\"query\\\": {\\n   \\\"multi_match\\\": {\\n    \\\"query\\\": \\\"orange dog\\\", \\n    \\\"fields\\\": [\\u201ctype^2\\\", \\\"color\\\"],\\n    \\\"type\\\": \\\"most_fields\\\"\\n  }\\n }\\n}<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
With the boost applied, the \\u201cbrown dog\\u201d documents now score 1.3 and come up above the \\u201corange cat.\\u201d<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
The lesson here is that searching across multiple fields can be tricky because the per-field scores are added together without concern for which fields are more important. To rectify this,<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
We can use boosting to express field priorities.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Query 7: dog<\\\/h4>\\n
<\\\/p>\\n
<\\\/p>\\n
Now we\\u2019re moving into advanced territory although our query looks simpler than anything we\\u2019ve seen before. We\\u2019re searching for \\u201cdog\\u201d and our index has three identical \\u201cdog\\u201d documents:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 1: \\\"dog\\\"\\nDoc 2: \\\"dog\\\"\\nDoc 3: \\\"dog\\\"<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
Which one is going to come up on top? You might guess that these three documents, being identical, should get the same score. So let\\u2019s take a moment to look at how ties are handled. When two documents have the same score, they\\u2019ll be sorted by their internal Lucene doc id. This internal id is different from the value in the document\\u2019s\\u00a0_id<\\\/strong>\\u00a0field, and it can differ even for the same document across replicas of a shard. If you really want ties to be broken in the same way regardless of which replica you hit, you can add a sort to your query, where you sort first by\\u00a0_score<\\\/strong>\\u00a0and then by a designated tiebreaker like _id or date.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
But this point about tiebreaking is only an aside. When I actually ran this query, the documents\\u00a0didn\\u2019t<\\\/em>\\u00a0come back with identical scores!<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n\\n\\n\\n\\n\\n\\n
ID<\\\/strong><\\\/td>\\nTitle<\\\/strong><\\\/td>\\nScore<\\\/strong><\\\/td>\\n<\\\/tr>\\n
1<\\\/td>\\ndog<\\\/td>\\n0.28<\\\/strong><\\\/td>\\n<\\\/tr>\\n
2<\\\/td>\\ndog<\\\/td>\\n0.18<\\\/td>\\n<\\\/tr>\\n
3<\\\/td>\\ndog<\\\/td>\\n0.18<\\\/td>\\n<\\\/tr>\\n<\\\/tbody>\\n<\\\/table>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
How is it possible that identical documents would get different scores? The lesson in this example is that<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
Term statistics are measured per shard.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Though I didn\\u2019t state it explicitly at the beginning of the post, all our previous examples were using one shard. In the current example, however, I set up my index with two shards. Document 1 landed on Shard 1 while Documents 2 and 3 landed on Shard 2. Document 1 got a higher score because within Shard 1, \\u201cdog\\u201d is a rarer term \\u2014 it only occurs once. Within Shard 2, \\u201cdog\\u201d is more common \\u2014 it occurs twice. Here\\u2019s how I set up the example:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
PUT \\\/test \\n{ \\\"settings\\\": { \\\"number_of_shards\\\": 2 } }\\n\\nPUT \\\/test\\\/_doc\\\/1?routing=0\\n{ \\\"title\\\" : \\\"dog\\\" } \\n\\nPUT \\\/test\\\/_doc\\\/2?routing=1\\n{ \\\"title\\\" : \\\"dog\\\" } \\n\\nPUT \\\/test\\\/_doc\\\/3?routing=1\\n{ \\\"title\\\" : \\\"dog\\\" } <\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
If you\\u2019re working with multiple shards and you want scores to be consistent regardless of which shard a document lives in, you can do a Distributed Frequency Search by added the following parameter to your query:\\u00a0search_type=dfs_query_then_fetch<\\\/strong>. This tells Elasticsearch to retrieve term statistics from all the shards and combine them before computing the scores.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
But its also important to know that<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
Replicas of a shard may have different term statistics.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
This is a consequence of how deletion is handled in Lucene. Documents that are marked for deletion but not yet physically removed (when their segments are merged) still contribute to term statistics. When a document is deleted, all the replicas will immediately \\u201cknow\\u201d about the deletion, but they might not carry it out physically at the same time, so they might end up with different term statistics. To reduce the impact of this, you can specify a user or session ID in the shard copy\\u00a0preference<\\\/strong>\\u00a0parameter. This encourages Elasticsearch to route requests from the same user to the same replicas, so that, for example, a user will not notice scoring discrepancies when issuing the same query multiple times.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Its also important to know that, from a scoring perspective,<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
Document updates behave like insertions, until segments are merged.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
When you update a document in Lucene, a new version of it is written to disk and the old version is marked for deletion. But the old version continues to contribute to term statistics until it is physically deleted.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
In the example below, I create a \\u201cdog cat\\u201d document and then I update its contents to be \\u201cdog zebra.\\u201d Immediately after the update, if I query for \\u201cdog\\u201d and look at the explain output, Elasticsearch tells me there are\\u00a0two\\u00a0<\\\/em>documents containing the term \\u201cdog.\\u201d The number goes down to one after I do a\\u00a0_forcemerge<\\\/strong>. The moral: if you\\u2019re doing relevancy tuning in Elasticsearch, looking closely at scores, and also updating documents at the same time, be sure to run a _forcemerge after your updates, or else rebuild the index entirely.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
PUT test\\\/_doc\\\/1\\n{ \\\"title\\\": \\\"dog cat\\\" }\\n\\nGET test\\\/_search?format=yaml\\n{ \\\"query\\\" : { \\\"match\\\" : { \\\"title\\\": \\\"dog\\\" } }, \\n  \\\"explain\\\": true }\\n\\nPUT test\\\/_doc\\\/1?refresh\\n{ \\\"title\\\": \\\"dog zebra\\\" }\\n\\nGET test\\\/_search?format=yaml\\n{ \\\"query\\\" : { \\\"match\\\" : { \\\"title\\\": \\\"dog\\\" } }, \\n  \\\"explain\\\": true }\\n\\nPOST test\\\/_forcemerge\\n\\nGET test\\\/_search?format=yaml\\n{ \\\"query\\\" : { \\\"match\\\" : { \\\"title\\\": \\\"dog\\\" } }, \\n  \\u201cexplain\\\": true }<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
Query 8: dog cat<\\\/h4>\\n
<\\\/p>\\n
<\\\/p>\\n
Now let\\u2019s take another look at Query 4, where we searched for \\u201cdog cat\\u201d and we found that a document containing both terms did better than documents with lots of instances of one term or the other. In Query 4, we were searching the title field, the only field available. Here, we\\u2019ve got\\u00a0two<\\\/em>\\u00a0fields, pet1 and pet2:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 1: {\\\"pet1\\\": \\\"dog\\\", \\\"pet2\\\": \\\"dog\\\"}\\nDoc 2: {\\\"pet1\\\": \\\"dog\\\", \\\"pet2\\\": \\\"cat\\\"}<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
We\\u2019ll do a multi_match including those two fields like this:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
GET test\\\/_search\\n{\\n  \\\"query\\\": {\\n   \\\"multi_match\\\": {\\n    \\\"query\\\": \\\"dog cat\\\", \\n    \\\"fields\\\": [\\u201cpet1\\u201d, \\\"pet2\\\"],\\n    \\\"type\\\": \\\"most_fields\\\"\\n  }\\n }\\n}<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
Since Document 2 matches both of our query terms we\\u2019d certainly hope that it does better than Document 1. That\\u2019s the lesson we took away from Query 4, right? Well, here are the results:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n\\n\\n\\n\\n\\n
ID<\\\/strong><\\\/td>\\nPet 1<\\\/strong><\\\/td>\\nPet 2<\\\/strong><\\\/td>\\nScore<\\\/strong><\\\/td>\\n<\\\/tr>\\n
1<\\\/td>\\ndog<\\\/td>\\ndog<\\\/td>\\n0.87<\\\/td>\\n<\\\/tr>\\n
2<\\\/td>\\ndog<\\\/td>\\ncat<\\\/td>\\n0.87<\\\/td>\\n<\\\/tr>\\n<\\\/tbody>\\n<\\\/table>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
You can see the documents are tied. It might look like \\u201ccat\\u201d is a rare term that should help Document 2 rise to the top, but within the Pet 2 field, \\u201ccat\\u201d and \\u201cdog\\u201d have the\\u00a0same<\\\/em>\\u00a0document frequencies, and the scoring is done on a per-field basis. It also looks like Document 2 should get an advantage for matching\\u00a0more<\\\/em>\\u00a0of the query terms, but again, the scoring is done a per-field basis: when we compute the score in the Pet 1 field, both documents do the same; when we compute the score in the Pet 2 field, both documents again do the same.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Does this contradict what we learned from Query 4? Not quite, but it warrants a refinement of the earlier lesson:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
Matching more query terms within the same field is good. But there\\u2019s no advantage when the matches happen across fields.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
If you\\u2019re not happy with this situation, there are some things you can do. You can combine the contents of Pet 1 and Pet 2 into a single field. You can also switch from the\\u00a0most_fields<\\\/strong>\\u00a0to the\\u00a0cross_fields<\\\/strong>\\u00a0query type to simulate a single field. (Just be aware that cross_fields has some other consequences on scoring, changing the procedure from field-centric to term-centric. We won\\u2019t go into details here.)<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Query 9: orange dog<\\\/h4>\\n
<\\\/p>\\n
<\\\/p>\\n
Now let\\u2019s revisit the lesson from Query 5, where we saw that matches in shorter fields are better. We\\u2019re going to search for \\u201corange dog.\\u201d We have a \\u201cdog\\u201d document with a description mentioning that the dog is brown. And we have a \\u201ccat\\u201d document with a description mentioning that the cat is sometimes orange. Notice that the dog document has a longer description than the cat document, and both descriptions are longer than the contents of the type field.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 1: {\\u201ctype\\u201d: \\u201cdog\\u201d, \\u201cdescription\\u201d: \\u201cA sweet and loving pet that is always eager to play. Brown coat. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non nibh sagittis, mollis ex a, scelerisque nisl. Ut vitae pellentesque magna, ut tristique nisi. Maecenas ut urna a elit posuere scelerisque. Suspendisse vel urna turpis. Mauris viverra fermentum ullamcorper. Duis ac lacus nibh. Nulla auctor lacus in purus vulputate, maximus ultricies augue scelerisque.\\u201d}\\n\\nDoc 2: {\\u201ctype\\u201d: \\u201ccat\\u201d, \\u201cdescription\\u201d: \\u201cPuzzlingly grumpy. Occasionally turns orange.\\u201d}<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
We\\u2019ll do a multi_match like this:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
GET test\\\/_search\\n{\\n  \\\"query\\\": {\\n   \\\"multi_match\\\": {\\n    \\\"query\\\": \\\"orange dog\\\", \\n    \\\"fields\\\": [\\\"type\\\", \\u201cdescription\\\"],\\n    \\\"type\\\": \\\"most_fields\\\"\\n  }\\n }\\n}<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
What\\u2019s going to take precedence here: the match for \\u201cdog\\u201d in the type field, which is a really short field, or the match for \\u201corange\\u201d in the description field, which is significantly longer? If matches in shorter fields are better, shouldn\\u2019t \\u201cdog\\u201d win here? In fact, the results look like this:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n\\n\\n\\n\\n\\n
ID<\\\/td>\\nType<\\\/strong><\\\/td>\\nDescription<\\\/strong><\\\/td>\\nScore<\\\/strong><\\\/td>\\n<\\\/tr>\\n
1<\\\/td>\\ndog<\\\/td>\\nA sweet\\u2026 brown\\u2026<\\\/td>\\n0.69<\\\/td>\\n<\\\/tr>\\n
2<\\\/td>\\ncat<\\\/td>\\nPuzzlingly grumpy\\u2026 orange.<\\\/td>\\n1.06<\\\/td>\\n<\\\/tr>\\n<\\\/tbody>\\n<\\\/table>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
The match for \\u201corange\\u201d in the Description field is sending Document 2 to the top even though that match occurs in a longer field body than the match for \\u201cdog\\u201d in Document 1. Does this contradict what we learned about matches in short and long fields from Query 5? No, but it points to something we hadn\\u2019t mentioned. The lesson is that:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
\\u201cShortness\\u201d is relative to the field\\u2019s average.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Within the type field, \\u201cdog\\u201d and \\u201ccat\\u201d don\\u2019t get an advantage for being short because, in fact, they\\u2019re both of average length for that field. On the other hand, the description for the cat is shorter than average for the description field overall, so it gets a benefit for being \\u201cshort.\\u201d<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Query 10: abcd efghijklmnopqrstuvwxyz<\\\/h4>\\n
<\\\/p>\\n
<\\\/p>\\n
Here\\u2019s an easy one to close out our set of examples. I\\u2019ve divided the alphabet into two query terms. One of them is a short term, \\u201cabcd,\\u201d and the other is a long term, \\u201cefghijklmnopqrstuvwxyz.\\u201d I\\u2019m going to search for both terms together: \\u201cabcd efghijklmnopqrstuvwxyz.\\u201d My index has one match for each term:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
Doc 1: \\\"abcd\\\"\\nDoc 2: \\\"efghijklmnopqrstuvwxyz\\\"<\\\/span><\\\/pre>\\n
<\\\/p>\\n
<\\\/p>\\n
Which document is going to do the best? The results look like this:<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n\\n\\n\\n\\n\\n
ID<\\\/td>\\nTitle<\\\/strong><\\\/td>\\nScore<\\\/strong><\\\/td>\\n<\\\/tr>\\n
1<\\\/td>\\nabcd<\\\/td>\\n0.69<\\\/td>\\n<\\\/tr>\\n
2<\\\/td>\\nefghijklmnopqrstuvwxyz<\\\/td>\\n0.69<\\\/td>\\n<\\\/tr>\\n<\\\/tbody>\\n<\\\/table>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
Why are the documents tied if it\\u2019s true that matches in shorter fields are better? The lesson is that<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
When we talk about a short or long field, we\\u2019re talking about how many\\u00a0terms<\\\/em>\\u00a0the field contains, not how many\\u00a0characters<\\\/em>. In this example, the titles for both documents are of length 1.<\\\/p>\\n
<\\\/p>\\n
<\\\/p>\\n
\\n
\\n
Term length is not significant.<\\\/p>\\n<\\\/blockquote>\\n<\\\/figure>\\n
<\\\/p>\\n
<\\\/p>\\n
That\\u2019s it for now. Hopefully these examples have helped build your intuitions about how scoring works in Elasticsearch and Solr. Thanks for following along! If you\\u2019d like to go further and understand how the BM25 scoring function actually achieves the behaviors we\\u2019ve seen here, check out our companion article on\\u00a0Understanding TF-IDF and BM25<\\\/a>.<\\\/p>\\n
<\\\/p>\"},\"elements\":[],\"widgetType\":\"text-editor\"}],\"isInner\":false}],\"isInner\":false},{\"id\":\"79f93a5\",\"elType\":\"section\",\"settings\":[],\"elements\":[{\"id\":\"c64b340\",\"elType\":\"column\",\"settings\":{\"_column_size\":100,\"_inline_size\":null,\"thegem_column_breakpoints_list\":[]},\"elements\":[{\"id\":\"064c386\",\"elType\":\"widget\",\"settings\":{\"prev_label\":\"Previous Post\",\"next_label\":\"Next Post\",\"show_borders\":\"\",\"title_typography_typography\":\"custom\",\"title_typography_font_size\":{\"unit\":\"px\",\"size\":14,\"sizes\":[]},\"title_typography_font_weight\":\"700\",\"__globals__\":{\"arrow_color\":\"globals\\\/colors?id=primary\",\"label_color\":\"globals\\\/colors?id=secondary\"},\"arrow\":\"fa fa-caret-left\",\"show_arrow\":\"\"},\"elements\":[],\"widgetType\":\"global\",\"templateID\":\"28083\"}],\"isInner\":false}],\"isInner\":false}]"],"thegem_post_page_elements_data":["a:12:{s:13:\"post_elements\";s:7:\"default\";s:11:\"show_author\";i:0;s:16:\"blog_hide_author\";i:0;s:14:\"blog_hide_date\";i:0;s:26:\"blog_hide_date_in_blog_cat\";i:0;s:20:\"blog_hide_categories\";i:0;s:14:\"blog_hide_tags\";i:0;s:18:\"blog_hide_comments\";i:0;s:15:\"blog_hide_likes\";i:0;s:20:\"blog_hide_navigation\";i:0;s:17:\"blog_hide_socials\";i:0;s:17:\"blog_hide_realted\";i:0;}"],"_last_editor_used_jetpack":["block-editor"],"_elementor_pro_version":["3.18.3"],"thegem_popups_data":["a:2:{s:20:\"popups_layout_source\";s:7:\"default\";s:12:\"thegemPopups\";a:0:{}}"],"_yoast_wpseo_primary_category":["36"],"_yoast_wpseo_content_score":["60"],"_yoast_wpseo_estimated-reading-time-minutes":["13"],"_yoast_wpseo_wordproof_timestamp":[""],"_elementor_controls_usage":["a:4:{s:11:\"text-editor\";a:3:{s:5:\"count\";i:1;s:15:\"control_percent\";i:0;s:8:\"controls\";a:1:{s:7:\"content\";a:1:{s:14:\"section_editor\";a:1:{s:6:\"editor\";i:1;}}}}s:6:\"column\";a:3:{s:5:\"count\";i:2;s:15:\"control_percent\";i:0;s:8:\"controls\";a:1:{s:6:\"layout\";a:1:{s:6:\"layout\";a:1:{s:12:\"_inline_size\";i:1;}}}}s:7:\"section\";a:3:{s:5:\"count\";i:2;s:15:\"control_percent\";i:0;s:8:\"controls\";a:0:{}}s:15:\"post-navigation\";a:3:{s:5:\"count\";i:1;s:15:\"control_percent\";i:2;s:8:\"controls\";a:2:{s:7:\"content\";a:1:{s:31:\"section_post_navigation_content\";a:5:{s:10:\"prev_label\";i:1;s:10:\"next_label\";i:1;s:12:\"show_borders\";i:1;s:5:\"arrow\";i:1;s:10:\"show_arrow\";i:1;}}s:5:\"style\";a:1:{s:11:\"title_style\";a:3:{s:27:\"title_typography_typography\";i:1;s:26:\"title_typography_font_size\";i:1;s:28:\"title_typography_font_weight\";i:1;}}}}}"],"_elementor_page_assets":["a:2:{s:7:\"scripts\";a:2:{i:0;s:18:\"elementor-frontend\";i:1;s:15:\"post-navigation\";}s:6:\"styles\";a:1:{i:0;s:22:\"widget-post-navigation\";}}"],"_elementor_css":["a:6:{s:4:\"time\";i:1775577129;s:5:\"fonts\";a:0:{}s:5:\"icons\";a:0:{}s:20:\"dynamic_elements_ids\";a:0:{}s:6:\"status\";s:4:\"file\";i:0;s:0:\"\";}"],"_elementor_element_cache":["{\"timeout\":1776314135,\"value\":{\"content\":\"\\t\\t
\\r\\n\\t\\t\\t\\t\\t\\t
\\r\\n\\t\\t\\t\\t\\t
\\n\\t\\t\\t
\\n\\t\\t\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjIyZDJhYWRjIiwiZWxUeXBlIjoid2lkZ2V0Iiwic2V0dGluZ3MiOnsiZWRpdG9yIjoiPHA+PCEtLSB3cDpoZWFkaW5nIHtcImxldmVsXCI6NH0gLS0+PFwvcD5cbjxwPjwhLS0gXC93cDpoZWFkaW5nIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+VGhlIGJ1aWx0LWluIHNjb3JpbmcgbWVjaGFuaXNtIGluIEVsYXN0aWNzZWFyY2ggYW5kIFNvbHIgY2FuIHNlZW0gbXlzdGVyaW91cyB0byBiZWdpbm5lcnMgYW5kIGV4cGVyaWVuY2VkIHByYWN0aXRpb25lcnMgYWxpa2UuIEluc3RlYWQgb2YgZGVsdmluZyBpbnRvIHRoZSBtYXRoZW1hdGljYWwgZGVmaW5pdGlvbnMgb2YgVEYtSURGIGFuZCBCTTI1LCB0aGlzIGFydGljbGUgd2lsbCBoZWxwIHlvdSBkZXZlbG9wIGFuIGludHVpdGl2ZSB1bmRlcnN0YW5kaW5nIG9mIHRoZXNlIG1ldHJpY3MgYnkgd2Fsa2luZyB5b3UgdGhyb3VnaCBhIHNlcmllcyBvZiBzaW1wbGUgZXhhbXBsZXMuIEVhY2ggZXhhbXBsZSBjb25zaXN0cyBvZiBhIHF1ZXJ5IGFuZCBsaXN0IG9mIHNldmVyYWwgaW5kZXhlZCBkb2N1bWVudHMuIEFzIHlvdSByZWFkIGFsb25nLCB0cnkgdG8gZ3Vlc3Mgd2hpY2ggZG9jdW1lbnQgY29tZXMgdXAgb24gdG9wIGZvciBlYWNoIHF1ZXJ5LiBJbiBlYWNoIGNhc2UsIHdlIHdpbGwgZXhhbWluZSB3aHkgdGhhdCBwYXJ0aWN1bGFyIGRvY3VtZW50IGdldHMgdGhlIGhpZ2hlc3Qgc2NvcmUgYW5kIHdlXHUyMDE5bGwgZXh0cmFjdCB0aGUgZ2VuZXJhbCBwcmluY2lwbGUgYmVoaW5kIHRoaXMgYmVoYXZpb3IuIEEgc2V0IG9mIHNpeCBleGFtcGxlcyB3aWxsIGJlIGZvbGxvd2VkIGJ5IGFuIGV4dHJhIGNyZWRpdCBzZWN0aW9uIGZvY3VzaW5nIG9uIG1vcmUgYWR2YW5jZWQgdG9waWNzLiBBbG9uZyB3aXRoIGlsbHVzdHJhdGluZyBhbGwgb2YgdGhlIGtleSBiZWhhdmlvcnMgb2YgQk0yNSwgb3VyIGV4YW1wbGVzIHdpbGwgdG91Y2ggb24gc29tZSBvZiB0aGUgZ290Y2hhcyBhcm91bmQgc2NvcmluZyBpbiBjbHVzdGVyIHNjZW5hcmlvLCB3aGVyZSBzaGFyZHMgYW5kIHJlcGxpY2FzIGNvbWUgaW50byBwbGF5LiBUaGlzIGFydGljbGUgYWltcyB0byB0ZWFjaCB5b3UsIGluIGEgc2hvcnQgdGltZSBhbmQgd2l0aG91dCBhbnkgbWF0aCwgZXZlcnl0aGluZyB5b3VcdTIwMTlsbCBldmVyIG5lZWQgdG8ga25vdyBhYm91dCBzY29yaW5nLiBIYXZpbmcgYSBzb2xpZCB1bmRlcnN0YW5kaW5nIG9mIHNjb3Jpbmcgd2lsbCBwcmVwYXJlIHlvdSB0byBiZXR0ZXIgZGlhZ25vc2UgcmVsZXZhbmNlIHByb2JsZW1zIGFuZCBpbXByb3ZlIHJlbGV2YW5jZSBpbiByZWFsLXdvcmxkIGFwcGxpY2F0aW9ucy48XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpoZWFkaW5nIHtcImxldmVsXCI6NH0gLS0+PFwvcD5cbjxoND5RdWVyeSAxOiBkb2c8XC9oND5cbjxwPjwhLS0gXC93cDpoZWFkaW5nIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+TGV0XHUyMDE5cyBzYXkgSSBzZWFyY2ggZm9yIFx1MjAxY2RvZ1x1MjAxZCBhbmQgdGhlcmUgYXJlIG9ubHkgdGhyZWUgZG9jdW1lbnRzIGluIG15IGluZGV4LCBhcyBzaG93biBiZWxvdy4gV2hpY2ggb25lIG9mIHRoZXNlIGRvY3VtZW50cyBpcyBnb2luZyB0byBjb21lIHVwIG9uIHRvcD88XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwcmUgY2xhc3M9XCJ3cC1ibG9jay1wcmVmb3JtYXR0ZWRcIj48c3BhbiBjbGFzcz1cImhhcy1pbmxpbmUtY29sb3IgaGFzLXZpdmlkLWN5YW4tYmx1ZS1jb2xvclwiPkRvYyAxOiBcImRvZ1wiXG5cbkRvYyAyOiBcImRvZyBkb2dcIlxuXG5Eb2MgMzogXCJkb2cgZG9nIGRvZzxcL3NwYW4+PFwvcHJlPlxuPHA+PCEtLSBcL3dwOnByZWZvcm1hdHRlZCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPklmIHlvdVx1MjAxOXJlIG5vdCBxdWl0ZSBzdXJlLCB0aGF0XHUyMDE5cyBnb29kLCBiZWNhdXNlIEkgaGF2ZW5cdTIwMTl0IGdpdmVuIHlvdSBlbm91Z2ggb2YgdGhlIGNvbnRleHQgdG8ga25vdyB0aGUgYW5zd2VyLiBBbGwgdGhlIHF1ZXJpZXMgaW4gdGhpcyBhcnRpY2xlIHdlcmUgdGVzdGVkIGluIEVsYXN0aWNzZWFyY2ggNy40IHdoZXJlIEJNMjUgaXMgdGhlIGRlZmF1bHQgc2NvcmluZyBhbGdvcml0aG0gYW5kIGl0cyBwYXJhbWV0ZXJzIGFyZSBzZXQgYXMgaz0xLjIgYW5kIGI9MC43LiAoUGxlYXNlIGlnbm9yZSB0aGF0IGlmIGl0XHUyMDE5cyBtZWFuaW5nbGVzcyB0byB5b3UuKSBJbiBtb3N0IG9mIHRoZSBleGFtcGxlcywgd2VcdTIwMTlsbCBhc3N1bWUgdGhlIGRvY3VtZW50cyBoYXZlIGEgc2luZ2xlIHRleHQgZmllbGQgY2FsbGVkIFx1MjAxY3RpdGxlXHUyMDFkIHRoYXQgdXNlcyB0aGUgc3RhbmRhcmQgYW5hbHl6ZXI6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5cIm1hcHBpbmdzXCI6IHtcbiAgICBcInByb3BlcnRpZXNcIjoge1xuICAgICAgXCJ0aXRsZVwiOiB7XG4gICAgICAgIFwidHlwZVwiOiBcInRleHRcIixcbiAgICAgICAgXCJhbmFseXplclwiOiBcInN0YW5kYXJkXCJcbiAgICAgIH1cbiAgICB9XG4gIH08XC9zcGFuPjxcL3ByZT5cbjxwPjwhLS0gXC93cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5Gb3IgdGhlIG1vc3QgcGFydCwgd2VcdTIwMTlsbCBiZSBkb2luZyBzaW1wbGVcdTAwYTA8c3Ryb25nPm1hdGNoPFwvc3Ryb25nPlx1MDBhMHF1ZXJpZXMgYWdhaW5zdCB0aGUgdGl0bGUgZmllbGQuIFJlY2FsbCB0aGF0IHRoZSBkZWZhdWx0IGJvb2xlYW4gb3BlcmF0b3IgaW4gRWxhc3RpY3NlYXJjaCBpc1x1MDBhMDxzdHJvbmc+T1I8XC9zdHJvbmc+LiBIZXJlXHUyMDE5cyB3aGF0IG91ciBcdTIwMWNkb2dcdTIwMWQgcXVlcnkgbG9va3MgbGlrZTo8XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwcmUgY2xhc3M9XCJ3cC1ibG9jay1wcmVmb3JtYXR0ZWRcIj48c3BhbiBjbGFzcz1cImhhcy1pbmxpbmUtY29sb3IgaGFzLXZpdmlkLWN5YW4tYmx1ZS1jb2xvclwiPkdFVCB0ZXN0XC9fc2VhcmNoXG57XG4gIFwicXVlcnlcIjoge1xuICAgICAgXCJtYXRjaFwiOiB7XG4gICAgICAgICAgXCJ0aXRsZVwiOiBcImRvZ1wiXG4gICAgICB9IFxuICB9XG59PFwvc3Bhbj48XC9wcmU+XG48cD48IS0tIFwvd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+V2l0aCB0aG9zZSBkZXRhaWxzIG91dCBvZiB0aGUgd2F5LCBhcmUgeW91IHJlYWR5IHRvIHRlbGwgbWUgd2hpY2ggb25lIG9mIHRob3NlIHRocmVlIGRvY3VtZW50cyAoXHUyMDFjZG9nLFx1MjAxZCBcdTIwMWNkb2cgZG9nLFx1MjAxZCBhbmQgXHUyMDFjZG9nIGRvZyBkb2dcdTIwMWQpIGlzIGdvaW5nIHRvIGdldCB0aGUgaGlnaGVzdCBzY29yZT88XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPkhlcmUgYXJlIHRoZSByZXN1bHRzOjxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnRhYmxlIHtcImhhc0ZpeGVkTGF5b3V0XCI6dHJ1ZX0gLS0+PFwvcD5cbjxmaWd1cmUgY2xhc3M9XCJ3cC1ibG9jay10YWJsZVwiPlxuPHRhYmxlIGNsYXNzPVwiaGFzLWZpeGVkLWxheW91dFwiPlxuPHRib2R5PlxuPHRyPlxuPHRkPjxzdHJvbmc+SUQ8XC9zdHJvbmc+PFwvdGQ+XG48dGQ+PHN0cm9uZz5UaXRsZTxcL3N0cm9uZz48XC90ZD5cbjx0ZD48c3Ryb25nPlNjb3JlPFwvc3Ryb25nPjxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+MTxcL3RkPlxuPHRkPmRvZzxcL3RkPlxuPHRkPjAuMTY3PFwvdGQ+XG48XC90cj5cbjx0cj5cbjx0ZD4yPFwvdGQ+XG48dGQ+ZG9nIGRvZzxcL3RkPlxuPHRkPjAuMTgzPFwvdGQ+XG48XC90cj5cbjx0cj5cbjx0ZD4zPFwvdGQ+XG48dGQ+ZG9nIGRvZyBkb2c8XC90ZD5cbjx0ZD48c3Ryb25nPjAuMTg5PFwvc3Ryb25nPjxcL3RkPlxuPFwvdHI+XG48XC90Ym9keT5cbjxcL3RhYmxlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnRhYmxlIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+RG9jIDMgZ2V0cyB0aGUgaGlnaGVzdCBzY29yZSBiZWNhdXNlIGl0IGhhcyBjb250YWlucyB0aGUgaGlnaGVzdCBudW1iZXIgb2YgdG9rZW5zIHRoYXQgbWF0Y2ggdGhlIHF1ZXJ5IHRlcm0uIEFub3RoZXIgd2F5IHRvIHNheSB0aGlzIGlzIHRoYXQgRG9jIDMgaGFzIHRoZSBoaWdoZXN0XHUwMGEwPHN0cm9uZz50ZXJtIGZyZXF1ZW5jeTxcL3N0cm9uZz5cdTAwYTBmb3IgdGhlIHRlcm0gaW4gcXVlc3Rpb24uIE9mIGNvdXJzZSBpZiB3ZSB3ZXJlIHVzaW5nIHRoZSBrZXl3b3JkIGFuYWx5emVyLCBvbmx5IERvYyAxIHdvdWxkIGhhdmUgbWF0Y2hlZCB0aGUgcXVlcnksIGJ1dCB3ZVx1MjAxOXJlIHVzaW5nIHRoZSBzdGFuZGFyZCBhbmFseXplciB3aGljaCBicmVha3MgdGhlIHRpdGxlIGludG8gbXVsdGlwbGUgdG9rZW5zLiBUaGUgbW9yYWwgb2YgdGhpcyBzdG9yeSBpcyB0aGF0LCBmcm9tIGEgc2NvcmluZyBwZXJzcGVjdGl2ZSBhdCBsZWFzdCw8XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwdWxscXVvdGUgLS0+PFwvcD5cbjxmaWd1cmUgY2xhc3M9XCJ3cC1ibG9jay1wdWxscXVvdGVcIj5cbjxibG9ja3F1b3RlPlxuPHA+SGlnaCB0ZXJtIGZyZXF1ZW5jeSBpcyBnb29kLjxcL3A+XG48XC9ibG9ja3F1b3RlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnB1bGxxdW90ZSAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPkJlZm9yZSB3ZSBtb3ZlIG9uLCB0YWtlIGEgbW9tZW50IHRvIGNvbXBhcmUgRG9jIDEgYW5kIERvYyAzLiBOb3RpY2UgdGhhdCBhbHRob3VnaCBEb2MgMyBoYXMgdGhyZWUgdGltZXMgdGhlIHRlcm0gZnJlcXVlbmN5IGZvciBcdTIwMWNkb2dcdTIwMWQgYXMgRG9jIDEsIGl0cyBzY29yZSBpc25cdTIwMTl0IHRocmVlIHRpbWVzIGFzIGhpZ2guIFNvIHdoaWxlIGhpZ2hlciB0ZXJtIGZyZXF1ZW5jeSBnaXZlcyBhIGhpZ2hlciBzY29yZSwgaXRzIGltcGFjdCBpcyBub3QgbXVsdGlwbGljYXRpdmUuPFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6aGVhZGluZyB7XCJsZXZlbFwiOjR9IC0tPjxcL3A+XG48aDQ+UXVlcnkgMjogZG9nIGRvZyBjYXQ8XC9oND5cbjxwPjwhLS0gXC93cDpoZWFkaW5nIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+Tm93IElcdTIwMTltIHNlYXJjaGluZyBmb3IgXHUyMDFjZG9nIGRvZyBjYXRcdTIwMWQgYW5kIHRoZXJlIGFyZSBvbmx5IHR3byBkb2N1bWVudHMgaW4gbXkgaW5kZXg6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5Eb2MgMTogXCJjYXRcIlxuRG9jIDI6IFwiZG9nXCI8XC9zcGFuPjxcL3ByZT5cbjxwPjwhLS0gXC93cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5XaGljaCBvbmUgb2YgdGhlc2UgaXMgZ29pbmcgdG8gY29tZSB1cCBvbiB0b3A\\\/IE9yIGFyZSB0aGV5IGdvaW5nIHRvIGJlIHRpZWQ\\\/PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5JbiBmYWN0LCBcdTIwMWNkb2dcdTIwMWQgaXMgdGhlIHdpbm5lciBoZXJlOjxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnRhYmxlIHtcImhhc0ZpeGVkTGF5b3V0XCI6dHJ1ZX0gLS0+PFwvcD5cbjxmaWd1cmUgY2xhc3M9XCJ3cC1ibG9jay10YWJsZVwiPlxuPHRhYmxlIGNsYXNzPVwiaGFzLWZpeGVkLWxheW91dFwiPlxuPHRib2R5PlxuPHRyPlxuPHRkPjxzdHJvbmc+SUQ8XC9zdHJvbmc+PFwvdGQ+XG48dGQ+PHN0cm9uZz5UaXRsZTxcL3N0cm9uZz48XC90ZD5cbjx0ZD48c3Ryb25nPlNjb3JlPFwvc3Ryb25nPjxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+MTxcL3RkPlxuPHRkPmNhdDxcL3RkPlxuPHRkPjAuNjxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+MjxcL3RkPlxuPHRkPmRvZzxcL3RkPlxuPHRkPjxzdHJvbmc+MS4zPFwvc3Ryb25nPjxcL3RkPlxuPFwvdHI+XG48XC90Ym9keT5cbjxcL3RhYmxlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnRhYmxlIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+V2h5IGRvZXMgXHUyMDFjZG9nXHUyMDFkIGdldCB0d2ljZSB0aGUgc2NvcmUgb2YgXHUyMDFjY2F0P1x1MjAxZCBUaGUgbGVzc29uIGhlcmUgaXMgdGhhdDxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnB1bGxxdW90ZSAtLT48XC9wPlxuPGZpZ3VyZSBjbGFzcz1cIndwLWJsb2NrLXB1bGxxdW90ZVwiPlxuPGJsb2NrcXVvdGU+XG48cD5TY29yZXMgZm9yIGVhY2ggcXVlcnkgdGVybSBhcmUgc3VtbWVkLjxcL3A+XG48XC9ibG9ja3F1b3RlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnB1bGxxdW90ZSAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPk91ciBxdWVyeSBoYXMgdHdvIGluc3RhbmNlcyBvZiBcdTIwMWNkb2cuXHUyMDFkIFRoZSBzY29yZSBmb3IgdGhlIHdob2xlIHF1ZXJ5IGlzIHRoZSBzdW0gb2YgdGhlIHNjb3JlcyBmb3IgZWFjaCB0ZXJtLiBFYWNoIGluc3RhbmNlIG9mIFx1MjAxY2RvZ1x1MjAxZCBpbiBvdXIgcXVlcnkgaXMgZ29pbmcgdG8gbWF0Y2ggdGhlIFx1MjAxY2RvZ1x1MjAxZCBpbiBEb2MgMiBhbmQgY29udHJpYnV0ZSByb3VnaGx5IDAuNiB0byB0aGUgc2NvcmUsIGZvciBhIHRvdGFsIG9mIDEuMy4gVXNpbmcgdGhlIHN0YW5kYXJkIGFuYWx5emVyLCBxdWVyeSB0ZXJtcyBhcmVuXHUyMDE5dCBkZWR1cGxpY2F0ZWQsIHNvIGVhY2ggaW5zdGFuY2Ugb2YgXHUyMDFjZG9nXHUyMDFkIGlzIHRyZWF0ZWQgc2VwYXJhdGVseS4gRG9jIDEgZG9lc25cdTIwMTl0IGdldCBhIHNpbWlsYXIgYWR2YW50YWdlIGJlY2F1c2Ugb3VyIHF1ZXJ5IG9ubHkgY29udGFpbnMgb25lIGluc3RhbmNlIG9mIFx1MjAxY2NhdC5cdTIwMWQ8XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpoZWFkaW5nIHtcImxldmVsXCI6NH0gLS0+PFwvcD5cbjxoND5RdWVyeSAzOiBkb2cgZG9nIGNhdDxcL2g0PlxuPHA+PCEtLSBcL3dwOmhlYWRpbmcgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5Ob3cgSVx1MjAxOW0gZXhlY3V0aW5nIHRoZSBzYW1lIHF1ZXJ5IGFzIGJlZm9yZSwgXHUyMDFjZG9nIGRvZyBjYXQsXHUyMDFkIGJ1dCBteSBpbmRleCBpcyBkaWZmZXJlbnQuIFRoaXMgdGltZSBJIGhhdmUgbG90cyBvZiBcdTIwMWNkb2dcdTIwMWQgZG9jdW1lbnRzOjxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnByZWZvcm1hdHRlZCAtLT48XC9wPlxuPHByZSBjbGFzcz1cIndwLWJsb2NrLXByZWZvcm1hdHRlZFwiPjxzcGFuIGNsYXNzPVwiaGFzLWlubGluZS1jb2xvciBoYXMtdml2aWQtY3lhbi1ibHVlLWNvbG9yXCI+RG9jIDE6IFwiZG9nXCJcbkRvYyAyOiBcImRvZ1wiXG5Eb2MgMzogXCJkb2dcIlxuRG9jIDQ6IFwiZG9nXCJcbkRvYyA1OiBcImRvZ1wiXG5Eb2MgNjogXCJkb2dcIlxuRG9jIDc6IFwiY2F0XCI8XC9zcGFuPjxcL3ByZT5cbjxwPjwhLS0gXC93cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5XaGF0XHUyMDE5cyBnb2luZyB0byBoYXBwZW4gbm93PyBEbyB0aGUgXHUyMDFjZG9nXHUyMDFkIGRvY3VtZW50cyBzdGlsbCB3aW4gb3ZlciBcdTIwMWNjYXRcdTIwMWQgYmVjYXVzZSBteSBxdWVyeSBtZW50aW9ucyBcdTIwMWNkb2dcdTIwMWQgdHdpY2U\\\/IEhlcmUgYXJlIHRoZSByZXN1bHRzOjxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnRhYmxlIHtcImhhc0ZpeGVkTGF5b3V0XCI6dHJ1ZX0gLS0+PFwvcD5cbjxmaWd1cmUgY2xhc3M9XCJ3cC1ibG9jay10YWJsZVwiPlxuPHRhYmxlIGNsYXNzPVwiaGFzLWZpeGVkLWxheW91dFwiPlxuPHRib2R5PlxuPHRyPlxuPHRkPjxzdHJvbmc+SUQ8XC9zdHJvbmc+PFwvdGQ+XG48dGQ+PHN0cm9uZz5UaXRsZTxcL3N0cm9uZz48XC90ZD5cbjx0ZD48c3Ryb25nPlNjb3JlPFwvc3Ryb25nPjxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+MTxcL3RkPlxuPHRkPmRvZzxcL3RkPlxuPHRkPjAuNDxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+MjxcL3RkPlxuPHRkPmRvZzxcL3RkPlxuPHRkPjAuNDxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+MzxcL3RkPlxuPHRkPmRvZzxcL3RkPlxuPHRkPjAuNDxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+NDxcL3RkPlxuPHRkPmRvZzxcL3RkPlxuPHRkPjAuNDxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+NTxcL3RkPlxuPHRkPmRvZzxcL3RkPlxuPHRkPjAuNDxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+NjxcL3RkPlxuPHRkPmRvZzxcL3RkPlxuPHRkPjAuNDxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+NzxcL3RkPlxuPHRkPmNhdDxcL3RkPlxuPHRkPjxzdHJvbmc+MS41PFwvc3Ryb25nPjxcL3RkPlxuPFwvdHI+XG48XC90Ym9keT5cbjxcL3RhYmxlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnRhYmxlIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+VGhlIHJlc3VsdHMgYXJlIGRpZmZlcmVudCB0aGlzIHRpbWUgYmVjYXVzZSB0aGUgdGVybXMgaGF2ZSBkaWZmZXJlbnRcdTAwYTA8c3Ryb25nPmRvY3VtZW50IGZyZXF1ZW5jaWVzPFwvc3Ryb25nPlx1MDBhMHRoYW4gYmVmb3JlLiBBIHRlcm1cdTIwMTlzIGRvY3VtZW50IGZyZXF1ZW5jeSBpcyB0aGUgbnVtYmVyIG9mIGRvY3VtZW50cyBpbiB0aGUgaW5kZXggdGhhdCBjb250YWluIHRoZSB0ZXJtLiBGcm9tIGEgc2NvcmluZyBwZXJzcGVjdGl2ZSwgbG93IGRvY3VtZW50IGZyZXF1ZW5jeSBpcyBnb29kIGFuZCBoaWdoIGRvY3VtZW50IGZyZXF1ZW5jeSBpcyBiYWQuIEluIHRoaXMgZXhhbXBsZSwgXHUyMDFjY2F0XHUyMDFkIGlzIGEgcmFyZSB0ZXJtIGluIHRoZSBpbmRleCBcdTIwMTQgaXQgaGFzIGxvdyBkb2N1bWVudCBmcmVxdWVuY3kgXHUyMDE0IHNvIG1hdGNoZXMgb24gdGhhdCB0ZXJtIGhlbHAgdGhlIHNjb3JlIG1vcmUgdGhhbiBtYXRjaGVzIG9uIFx1MjAxY2RvZyxcdTIwMWQgd2hpY2ggaXMgYSBjb21tb24gdGVybS4gVGhlIGxlc3NvbiBoZXJlIGlzIHRoYXQ6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48ZmlndXJlIGNsYXNzPVwid3AtYmxvY2stcHVsbHF1b3RlXCI+XG48YmxvY2txdW90ZT5cbjxwPk1hdGNoZXMgZm9yIHJhcmVyIHRlcm1zIGFyZSBiZXR0ZXIuPFwvcD5cbjxcL2Jsb2NrcXVvdGU+XG48XC9maWd1cmU+XG48cD48IS0tIFwvd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+SWYgSSB3YW50IHRvIHRlbGwgdGhlIHNlYXJjaCBlbmdpbmUgdGhhdCBhIGNvbW1vbiB0ZXJtIGlzIHBhcnRpY3VsYXJseSBpbXBvcnRhbnQgdG8gbWUgaW4gYSBjZXJ0YWluIHNjZW5hcmlvLCBJIGNhblx1MDBhMDxzdHJvbmc+Ym9vc3Q8XC9zdHJvbmc+XHUwMGEwdGhlIHRlcm0uIElmIEkgaGFkIGV4ZWN1dGVkIG15IHF1ZXJ5IHdpdGggYSBib29zdCBvZiA3IG9uIFx1MjAxY2RvZyxcdTIwMWQgdGhlIGRvZyBkb2N1bWVudHMgd291bGQgY29tZSB1cCBhYm92ZSB0aGUgY2F0IGRvY3VtZW50LiBIZXJlXHUyMDE5cyBob3cgSVx1MjAxOWQgc2V0IHRoYXQgdXA6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5HRVQgdGVzdFwvX3NlYXJjaFxue1xuIFwicXVlcnlcIjoge1xuICBcInF1ZXJ5X3N0cmluZ1wiOiB7XG4gICBcInF1ZXJ5XCI6IFwiZG9nXjcgY2F0XCIsXG4gICBcImZpZWxkc1wiOiBbXCJ0aXRsZVwiXVxuICB9IFxuIH1cbn08XC9zcGFuPjxcL3ByZT5cbjxwPjwhLS0gXC93cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwPjwhLS0gd3A6aGVhZGluZyB7XCJsZXZlbFwiOjR9IC0tPjxcL3A+XG48aDQ+UXVlcnkgNDogZG9nIGNhdDxcL2g0PlxuPHA+PCEtLSBcL3dwOmhlYWRpbmcgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5JbiB0aGlzIGV4YW1wbGUgSVx1MjAxOW0gc2VhcmNoaW5nIGZvciBcdTIwMWNkb2cgY2F0LFx1MjAxZCBhbmQgSVx1MjAxOXZlIGdvdCB0aHJlZSBkb2N1bWVudHMgaW4gbXkgaW5kZXg6IG9uZSB3aXRoIGEgbG90IG9mIGRvZ3MsIG9uZSB3aXRoIGEgbG90IG9mIGNhdHMsIGFuZCBvbmUgd2l0aCBhIHNpbmdsZSBpbnN0YW5jZSBvZiBkb2cgYW5kIGNhdCBlYWNoLCBwbHVzIGEgbG90IG9mIG1hdHMuPFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5Eb2MgMTogXCJkb2cgZG9nIGRvZyBkb2cgZG9nIGRvZyBkb2dcIlxuRG9jIDI6IFwiY2F0IGNhdCBjYXQgY2F0IGNhdCBjYXQgY2F0XCJcbkRvYyAzOiBcImRvZyBjYXQgbWF0IG1hdCBtYXQgbWF0IG1hdFwiPFwvc3Bhbj48XC9wcmU+XG48cD48IS0tIFwvd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+V2hpY2ggZG9jdW1lbnQgY29tZXMgdXAgb24gdG9wIHRoaXMgdGltZT8gTm90aWNlIHRoYXQgaW4gRG9jIDEgYW5kIERvYyAyLCBldmVyeSBzaW5nbGUgdGVybSBtYXRjaGVzIG9uZSBvZiB0aGUgcXVlcnkgdGVybXMsIHdoZXJlYXMgaW4gRG9jIDMgdGhlcmUgYXJlIGZpdmUgdGVybXMgdGhhdCBkb25cdTIwMTl0IG1hdGNoIGFueXRoaW5nLiBTbyB0aGUgcmVzdWx0cyBtaWdodCBiZSBhIGxpdHRsZSBzdXJwcmlzaW5nOjxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnRhYmxlIHtcImhhc0ZpeGVkTGF5b3V0XCI6dHJ1ZX0gLS0+PFwvcD5cbjxmaWd1cmUgY2xhc3M9XCJ3cC1ibG9jay10YWJsZVwiPlxuPHRhYmxlIGNsYXNzPVwiaGFzLWZpeGVkLWxheW91dFwiPlxuPHRib2R5PlxuPHRyPlxuPHRkPklEPFwvdGQ+XG48dGQ+VGl0bGU8XC90ZD5cbjx0ZD5TY29yZTxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+MTxcL3RkPlxuPHRkPmRvZyBkb2cgZG9nIGRvZyBkb2cgZG9nIGRvZzxcL3RkPlxuPHRkPjAuODg8XC90ZD5cbjxcL3RyPlxuPHRyPlxuPHRkPjI8XC90ZD5cbjx0ZD5jYXQgY2F0IGNhdCBjYXQgY2F0IGNhdCBjYXQ8XC90ZD5cbjx0ZD4wLjg4PFwvdGQ+XG48XC90cj5cbjx0cj5cbjx0ZD4zPFwvdGQ+XG48dGQ+ZG9nIGNhdCBtYXQgbWF0IG1hdCBtYXQgbWF0PFwvdGQ+XG48dGQ+PHN0cm9uZz4wLjk0PFwvc3Ryb25nPjxcL3RkPlxuPFwvdHI+XG48XC90Ym9keT5cbjxcL3RhYmxlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnRhYmxlIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+RG9jdW1lbnQgMyBnZXRzIHRoZSBoaWdoZXN0IHNjb3JlIGJlY2F1c2UgaXQgaGFzIG1hdGNoZXMgZm9yXHUwMGEwPGVtPmJvdGg8XC9lbT5cdTAwYTBvZiB0aGUgcXVlcnkgdGVybXMsIFx1MjAxY2RvZ1x1MjAxZCBhbmQgXHUyMDFjY2F0Llx1MjAxZCBXaGlsZSBEb2N1bWVudHMgMSBhbmQgMiBoYXZlIGhpZ2hlciB0ZXJtIGZyZXF1ZW5jeSBmb3IgXHUyMDFjZG9nXHUyMDFkIGFuZCBcdTIwMWNjYXRcdTIwMWQgcmVzcGVjdGl2ZWx5LCB0aGV5IGVhY2ggY29udGFpbiBvbmx5IG9uZSBvZiB0aGUgdGVybXMuIFRoZSBsZXNzb24gaXMgdGhhdDxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnB1bGxxdW90ZSAtLT48XC9wPlxuPGZpZ3VyZSBjbGFzcz1cIndwLWJsb2NrLXB1bGxxdW90ZVwiPlxuPGJsb2NrcXVvdGU+XG48cD5NYXRjaGluZyBtb3JlIHF1ZXJ5IHRlcm1zIGlzIGdvb2QuPFwvcD5cbjxcL2Jsb2NrcXVvdGU+XG48XC9maWd1cmU+XG48cD48IS0tIFwvd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48cD48IS0tIHdwOmhlYWRpbmcge1wibGV2ZWxcIjo0fSAtLT48XC9wPlxuPGg0PlF1ZXJ5IDU6IGRvZzxcL2g0PlxuPHA+PCEtLSBcL3dwOmhlYWRpbmcgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5Ob3cgSVx1MjAxOWxsIHNlYXJjaCBmb3IgXHUyMDFjZG9nXHUyMDFkIGFuZCB0aGVyZSBhcmUgb25seSB0d28gZG9jdW1lbnRzIGluIG15IGluZGV4OjxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnByZWZvcm1hdHRlZCAtLT48XC9wPlxuPHByZSBjbGFzcz1cIndwLWJsb2NrLXByZWZvcm1hdHRlZFwiPjxzcGFuIGNsYXNzPVwiaGFzLWlubGluZS1jb2xvciBoYXMtdml2aWQtY3lhbi1ibHVlLWNvbG9yXCI+RG9jIDE6IFwiZG9nIGNhdCB6ZWJyYVwiXG5Eb2MgMjogXCJkb2cgY2F0PFwvc3Bhbj48XC9wcmU+XG48cD48IS0tIFwvd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+Qm90aCBvZiB0aGVzZSBkb2N1bWVudHMgbWF0Y2ggbXkgcXVlcnksIGFuZCBib3RoIGhhdmUgc29tZSB0ZXJtcyB0aGF0IGRvblx1MjAxOXQgbWF0Y2guIFdoaWNoIGRvY3VtZW50IGRvZXMgdGhlIGJlc3Q\\\/IEhlcmUgYXJlIHRoZXJlIHJlc3VsdHM6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6dGFibGUge1wiaGFzRml4ZWRMYXlvdXRcIjp0cnVlfSAtLT48XC9wPlxuPGZpZ3VyZSBjbGFzcz1cIndwLWJsb2NrLXRhYmxlXCI+XG48dGFibGUgY2xhc3M9XCJoYXMtZml4ZWQtbGF5b3V0XCI+XG48dGJvZHk+XG48dHI+XG48dGQ+SUQ8XC90ZD5cbjx0ZD5UaXRsZTxcL3RkPlxuPHRkPlNjb3JlPFwvdGQ+XG48XC90cj5cbjx0cj5cbjx0ZD4xPFwvdGQ+XG48dGQ+ZG9nIGNhdCB6ZWJyYTxcL3RkPlxuPHRkPjAuMTY8XC90ZD5cbjxcL3RyPlxuPHRyPlxuPHRkPjI8XC90ZD5cbjx0ZD5kb2cgY2F0PFwvdGQ+XG48dGQ+PHN0cm9uZz4wLjE5PFwvc3Ryb25nPjxcL3RkPlxuPFwvdHI+XG48XC90Ym9keT5cbjxcL3RhYmxlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnRhYmxlIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+SW4gdGhpcyBjYXNlLCBEb2N1bWVudCAyIGRvZXMgYmV0dGVyIGJlY2F1c2UgaXQgaXMgc2hvcnRlci4gVGhlIHRoaW5raW5nIGlzIHRoYXQgd2hlbiBhIHRlcm0gb2NjdXJzIGluIGEgc2hvcnRlciBkb2N1bWVudCwgd2UgY2FuIGJlIG1vcmUgY29uZmlkZW50IHRoYXQgdGhlIHRlcm0gaXMgc2lnbmlmaWNhbnQgdG8gdGhlIGRvY3VtZW50IChvciB0aGF0IHRoZSBkb2N1bWVudCBpc1x1MDBhMDxlbT5hYm91dDxcL2VtPlx1MDBhMHRoZSB0ZXJtKS4gV2hlbiBhIHRlcm0gb2NjdXJzIGluIGEgbG9uZ2VyIGRvY3VtZW50LCB3ZSBoYXZlIGxlc3MgY29uZmlkZW5jZSB0aGF0IHRoaXMgb2NjdXJyZW5jZSBpcyBtZWFuaW5nZnVsLiBUaGUgbGVzc29uIGhlcmUgaXMgdGhhdDxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnB1bGxxdW90ZSAtLT48XC9wPlxuPGZpZ3VyZSBjbGFzcz1cIndwLWJsb2NrLXB1bGxxdW90ZVwiPlxuPGJsb2NrcXVvdGU+XG48cD5NYXRjaGVzIGluIHNob3J0ZXIgZG9jdW1lbnRzIGFyZSBiZXR0ZXIuPFwvcD5cbjxcL2Jsb2NrcXVvdGU+XG48XC9maWd1cmU+XG48cD48IS0tIFwvd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48cD48IS0tIHdwOmhlYWRpbmcge1wibGV2ZWxcIjo0fSAtLT48XC9wPlxuPGg0PlF1ZXJ5IDY6IG9yYW5nZSBkb2c8XC9oND5cbjxwPjwhLS0gXC93cDpoZWFkaW5nIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+Tm93IGxldFx1MjAxOXMgY29uc2lkZXIgYSBzY2VuYXJpbyB0aGF0XHUyMDE5cyBhIGxpdHRsZSBtb3JlIGNvbXBsaWNhdGVkIHRoYW4gdGhlIHByZXZpb3VzIG9uZXMuIEZvciB0aGUgZmlyc3QgdGltZSwgb3VyIGRvY3VtZW50cyB3aWxsIGhhdmUgdHdvIGZpZWxkcywgXHUyMDFjY29sb3JcdTIwMWQgYW5kIFx1MjAxY3R5cGUuXHUyMDFkPFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5Eb2MgMToge1wiY29sb3JcIjogXCJicm93blwiLCBcInR5cGVcIjogXCJkb2dcIn1cbkRvYyAyOiB7XCJjb2xvclwiOiBcImJyb3duXCIsIFwidHlwZVwiOiBcImRvZ1wifVxuRG9jIDM6IHtcImNvbG9yXCI6IFwiYnJvd25cIiwgXCJ0eXBlXCI6IFwiY2F0XCJ9XG5Eb2MgNDoge1wiY29sb3JcIjogXCJvcmFuZ2VcIiwgXCJ0eXBlXCI6IFwiY2F0XCJ9PFwvc3Bhbj48XC9wcmU+XG48cD48IS0tIFwvd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+V2VcdTIwMTlyZSBzZWFyY2hpbmcgZm9yIFx1MjAxY29yYW5nZSBkb2dcdTIwMWQgYnV0IGFzIHlvdSBjYW4gc2VlLCB0aGVyZSBhcmUgbm8gb3JhbmdlIGRvZ3MgaW4gdGhlIGluZGV4LiBUaGVyZSBhcmUgdHdvIGJyb3duIGRvZ3MsIGEgYnJvd24gY2F0LCBhbmQgYW4gb3JhbmdlIGNhdC4gV2hpY2ggb25lIGlzIGdvaW5nIHRvIGNvbWUgdXAgb24gdG9wPzxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+SSBzaG91bGQgbWVudGlvbiB0aGF0IHdlXHUyMDE5cmUgc2VhcmNoaW5nIGFjcm9zcyBib3RoIGZpZWxkcyB1c2luZyBhXHUwMGEwPHN0cm9uZz5tdWx0aV9tYXRjaDxcL3N0cm9uZz5cdTAwYTBsaWtlIHRoaXM6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5HRVQgdGVzdFwvX3NlYXJjaFxue1xuICBcInF1ZXJ5XCI6IHtcbiAgIFwibXVsdGlfbWF0Y2hcIjoge1xuICAgIFwicXVlcnlcIjogXCJvcmFuZ2UgZG9nXCIsIFxuICAgIFwiZmllbGRzXCI6IFtcInR5cGVcIiwgXCJjb2xvclwiXSxcbiAgICBcInR5cGVcIjogXCJtb3N0X2ZpZWxkc1wiXG4gIH1cbiB9XG59PFwvc3Bhbj48XC9wcmU+XG48cD48IS0tIFwvd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+SGVyZSBhcmUgdGhlIHJlc3VsdHM6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6dGFibGUge1wiaGFzRml4ZWRMYXlvdXRcIjp0cnVlfSAtLT48XC9wPlxuPGZpZ3VyZSBjbGFzcz1cIndwLWJsb2NrLXRhYmxlXCI+XG48dGFibGUgY2xhc3M9XCJoYXMtZml4ZWQtbGF5b3V0XCI+XG48dGJvZHk+XG48dHI+XG48dGQ+SUQ8XC90ZD5cbjx0ZD5Db2xvcjxcL3RkPlxuPHRkPlR5cGU8XC90ZD5cbjx0ZD5TY29yZTxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+MTxcL3RkPlxuPHRkPmJyb3duPFwvdGQ+XG48dGQ+ZG9nPFwvdGQ+XG48dGQ+MC42PFwvdGQ+XG48XC90cj5cbjx0cj5cbjx0ZD4yPFwvdGQ+XG48dGQ+YnJvd248XC90ZD5cbjx0ZD5kb2c8XC90ZD5cbjx0ZD4wLjY8XC90ZD5cbjxcL3RyPlxuPHRyPlxuPHRkPjM8XC90ZD5cbjx0ZD5icm93bjxcL3RkPlxuPHRkPmNhdDxcL3RkPlxuPHRkPk5cL0E8XC90ZD5cbjxcL3RyPlxuPHRyPlxuPHRkPjQ8XC90ZD5cbjx0ZD5vcmFuZ2U8XC90ZD5cbjx0ZD5jYXQ8XC90ZD5cbjx0ZD4xLjI8XC90ZD5cbjxcL3RyPlxuPFwvdGJvZHk+XG48XC90YWJsZT5cbjxcL2ZpZ3VyZT5cbjxwPjwhLS0gXC93cDp0YWJsZSAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPlRoaXMgZXhhbXBsZSBoaW50cyBhdCBzb21lIG9mIHRoZSB1bmV4cGVjdGVkIGJlaGF2aW9yIHRoYXQgY2FuIGFyaXNlIHdoZW4gd2Ugc2VhcmNoIGFjcm9zcyBtdWx0aXBsZSBmaWVsZHMuIFRoZSBzZWFyY2ggZW5naW5lIGRvZXNuXHUyMDE5dCBrbm93IHdoaWNoIGZpZWxkIGlzIG1vc3QgaW1wb3J0YW50IHRvIHVzLiBJZiBzb21lb25lIGlzIHNlYXJjaGluZyBmb3IgYW4gb3JhbmdlIGRvZywgd2UgbWlnaHQgZ3Vlc3MgdGhleVx1MjAxOXJlIG1vcmUgaW50ZXJlc3RlZCBpbiBzZWVpbmcgZG9ncyB0aGFuIHNlZWluZyBhcmJpdHJhcnkgdGhpbmdzIHRoYXQgaGFwcGVuIHRvIGJlIG9yYW5nZS4gKFx1MjAxY09yYW5nZSBkb2dcdTIwMWQgd291bGQgYmUgdmVyeSBzdHJhbmdlIHF1ZXJ5IHRvIGVudGVyIGlmIHlvdSBtZWFudCBcdTIwMWNzaG93IG1lIGFueXRoaW5nIHRoYXRcdTIwMTlzIG9yYW5nZS5cdTIwMWQpIEluIHRoaXMgY2FzZSwgaG93ZXZlciwgdGhlIGNvbG9yIGZpZWxkIGlzIHRha2luZyBwcmlvcml0eSBiZWNhdXNlIFx1MjAxY29yYW5nZVx1MjAxZCBpcyBhIHJhcmUgdGVybSB3aXRoaW4gdGhhdCBmaWVsZCAodGhlcmUgYXJlIDMgYnJvd25zIGFuZCBvbmx5IDEgb3JhbmdlKS4gV2l0aGluIHRoZSB0eXBlIGZpZWxkLCBcdTIwMWNkb2dcdTIwMWQgYW5kIFx1MjAxY2NhdFx1MjAxZCBoYXZlIHRoZSBzYW1lIGZyZXF1ZW5jeS4gVGhlIG9yYW5nZSBjYXQgY29tZXMgdXAgb24gdG9wIGJlY2F1c2UgYSBtYXRjaCBmb3IgdGhlIHJhcmUgdGVybSBcdTIwMWNvcmFuZ2VcdTIwMWQgaXMgdHJlYXRlZCBhcyBtb3JlIHZhbHVhYmxlIHRoYW4gYSBtYXRjaCBmb3IgXHUyMDFjZG9nLlx1MjAxZDxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+SWYgd2Ugd2FudCB0byBnaXZlIG1vcmUgd2VpZ2h0IHRvIHRoZSBcdTIwMWN0eXBlXHUyMDFkIGZpZWxkIHdlIGNhbiBib29zdCBpdCBsaWtlIHRoaXM6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5HRVQgdGVzdFwvX3NlYXJjaFxue1xuICBcInF1ZXJ5XCI6IHtcbiAgIFwibXVsdGlfbWF0Y2hcIjoge1xuICAgIFwicXVlcnlcIjogXCJvcmFuZ2UgZG9nXCIsIFxuICAgIFwiZmllbGRzXCI6IFtcdTIwMWN0eXBlXjJcIiwgXCJjb2xvclwiXSxcbiAgICBcInR5cGVcIjogXCJtb3N0X2ZpZWxkc1wiXG4gIH1cbiB9XG59PFwvc3Bhbj48XC9wcmU+XG48cD48IS0tIFwvd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+V2l0aCB0aGUgYm9vc3QgYXBwbGllZCwgdGhlIFx1MjAxY2Jyb3duIGRvZ1x1MjAxZCBkb2N1bWVudHMgbm93IHNjb3JlIDEuMyBhbmQgY29tZSB1cCBhYm92ZSB0aGUgXHUyMDFjb3JhbmdlIGNhdC5cdTIwMWQ8XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPlRoZSBsZXNzb24gaGVyZSBpcyB0aGF0IHNlYXJjaGluZyBhY3Jvc3MgbXVsdGlwbGUgZmllbGRzIGNhbiBiZSB0cmlja3kgYmVjYXVzZSB0aGUgcGVyLWZpZWxkIHNjb3JlcyBhcmUgYWRkZWQgdG9nZXRoZXIgd2l0aG91dCBjb25jZXJuIGZvciB3aGljaCBmaWVsZHMgYXJlIG1vcmUgaW1wb3J0YW50LiBUbyByZWN0aWZ5IHRoaXMsPFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48ZmlndXJlIGNsYXNzPVwid3AtYmxvY2stcHVsbHF1b3RlXCI+XG48YmxvY2txdW90ZT5cbjxwPldlIGNhbiB1c2UgYm9vc3RpbmcgdG8gZXhwcmVzcyBmaWVsZCBwcmlvcml0aWVzLjxcL3A+XG48XC9ibG9ja3F1b3RlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnB1bGxxdW90ZSAtLT48XC9wPlxuPHA+PCEtLSB3cDpoZWFkaW5nIHtcImxldmVsXCI6NH0gLS0+PFwvcD5cbjxoND5RdWVyeSA3OiBkb2c8XC9oND5cbjxwPjwhLS0gXC93cDpoZWFkaW5nIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+Tm93IHdlXHUyMDE5cmUgbW92aW5nIGludG8gYWR2YW5jZWQgdGVycml0b3J5IGFsdGhvdWdoIG91ciBxdWVyeSBsb29rcyBzaW1wbGVyIHRoYW4gYW55dGhpbmcgd2VcdTIwMTl2ZSBzZWVuIGJlZm9yZS4gV2VcdTIwMTlyZSBzZWFyY2hpbmcgZm9yIFx1MjAxY2RvZ1x1MjAxZCBhbmQgb3VyIGluZGV4IGhhcyB0aHJlZSBpZGVudGljYWwgXHUyMDFjZG9nXHUyMDFkIGRvY3VtZW50czo8XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwcmUgY2xhc3M9XCJ3cC1ibG9jay1wcmVmb3JtYXR0ZWRcIj48c3BhbiBjbGFzcz1cImhhcy1pbmxpbmUtY29sb3IgaGFzLXZpdmlkLWN5YW4tYmx1ZS1jb2xvclwiPkRvYyAxOiBcImRvZ1wiXG5Eb2MgMjogXCJkb2dcIlxuRG9jIDM6IFwiZG9nXCI8XC9zcGFuPjxcL3ByZT5cbjxwPjwhLS0gXC93cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5XaGljaCBvbmUgaXMgZ29pbmcgdG8gY29tZSB1cCBvbiB0b3A\\\/IFlvdSBtaWdodCBndWVzcyB0aGF0IHRoZXNlIHRocmVlIGRvY3VtZW50cywgYmVpbmcgaWRlbnRpY2FsLCBzaG91bGQgZ2V0IHRoZSBzYW1lIHNjb3JlLiBTbyBsZXRcdTIwMTlzIHRha2UgYSBtb21lbnQgdG8gbG9vayBhdCBob3cgdGllcyBhcmUgaGFuZGxlZC4gV2hlbiB0d28gZG9jdW1lbnRzIGhhdmUgdGhlIHNhbWUgc2NvcmUsIHRoZXlcdTIwMTlsbCBiZSBzb3J0ZWQgYnkgdGhlaXIgaW50ZXJuYWwgTHVjZW5lIGRvYyBpZC4gVGhpcyBpbnRlcm5hbCBpZCBpcyBkaWZmZXJlbnQgZnJvbSB0aGUgdmFsdWUgaW4gdGhlIGRvY3VtZW50XHUyMDE5c1x1MDBhMDxzdHJvbmc+X2lkPFwvc3Ryb25nPlx1MDBhMGZpZWxkLCBhbmQgaXQgY2FuIGRpZmZlciBldmVuIGZvciB0aGUgc2FtZSBkb2N1bWVudCBhY3Jvc3MgcmVwbGljYXMgb2YgYSBzaGFyZC4gSWYgeW91IHJlYWxseSB3YW50IHRpZXMgdG8gYmUgYnJva2VuIGluIHRoZSBzYW1lIHdheSByZWdhcmRsZXNzIG9mIHdoaWNoIHJlcGxpY2EgeW91IGhpdCwgeW91IGNhbiBhZGQgYSBzb3J0IHRvIHlvdXIgcXVlcnksIHdoZXJlIHlvdSBzb3J0IGZpcnN0IGJ5XHUwMGEwPHN0cm9uZz5fc2NvcmU8XC9zdHJvbmc+XHUwMGEwYW5kIHRoZW4gYnkgYSBkZXNpZ25hdGVkIHRpZWJyZWFrZXIgbGlrZSBfaWQgb3IgZGF0ZS48XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPkJ1dCB0aGlzIHBvaW50IGFib3V0IHRpZWJyZWFraW5nIGlzIG9ubHkgYW4gYXNpZGUuIFdoZW4gSSBhY3R1YWxseSByYW4gdGhpcyBxdWVyeSwgdGhlIGRvY3VtZW50c1x1MDBhMDxlbT5kaWRuXHUyMDE5dDxcL2VtPlx1MDBhMGNvbWUgYmFjayB3aXRoIGlkZW50aWNhbCBzY29yZXMhPFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6dGFibGUge1wiaGFzRml4ZWRMYXlvdXRcIjp0cnVlfSAtLT48XC9wPlxuPGZpZ3VyZSBjbGFzcz1cIndwLWJsb2NrLXRhYmxlXCI+XG48dGFibGUgY2xhc3M9XCJoYXMtZml4ZWQtbGF5b3V0XCI+XG48dGJvZHk+XG48dHI+XG48dGQ+PHN0cm9uZz5JRDxcL3N0cm9uZz48XC90ZD5cbjx0ZD48c3Ryb25nPlRpdGxlPFwvc3Ryb25nPjxcL3RkPlxuPHRkPjxzdHJvbmc+U2NvcmU8XC9zdHJvbmc+PFwvdGQ+XG48XC90cj5cbjx0cj5cbjx0ZD4xPFwvdGQ+XG48dGQ+ZG9nPFwvdGQ+XG48dGQ+PHN0cm9uZz4wLjI4PFwvc3Ryb25nPjxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+MjxcL3RkPlxuPHRkPmRvZzxcL3RkPlxuPHRkPjAuMTg8XC90ZD5cbjxcL3RyPlxuPHRyPlxuPHRkPjM8XC90ZD5cbjx0ZD5kb2c8XC90ZD5cbjx0ZD4wLjE4PFwvdGQ+XG48XC90cj5cbjxcL3Rib2R5PlxuPFwvdGFibGU+XG48XC9maWd1cmU+XG48cD48IS0tIFwvd3A6dGFibGUgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5Ib3cgaXMgaXQgcG9zc2libGUgdGhhdCBpZGVudGljYWwgZG9jdW1lbnRzIHdvdWxkIGdldCBkaWZmZXJlbnQgc2NvcmVzPyBUaGUgbGVzc29uIGluIHRoaXMgZXhhbXBsZSBpcyB0aGF0PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48ZmlndXJlIGNsYXNzPVwid3AtYmxvY2stcHVsbHF1b3RlXCI+XG48YmxvY2txdW90ZT5cbjxwPlRlcm0gc3RhdGlzdGljcyBhcmUgbWVhc3VyZWQgcGVyIHNoYXJkLjxcL3A+XG48XC9ibG9ja3F1b3RlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnB1bGxxdW90ZSAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPlRob3VnaCBJIGRpZG5cdTIwMTl0IHN0YXRlIGl0IGV4cGxpY2l0bHkgYXQgdGhlIGJlZ2lubmluZyBvZiB0aGUgcG9zdCwgYWxsIG91ciBwcmV2aW91cyBleGFtcGxlcyB3ZXJlIHVzaW5nIG9uZSBzaGFyZC4gSW4gdGhlIGN1cnJlbnQgZXhhbXBsZSwgaG93ZXZlciwgSSBzZXQgdXAgbXkgaW5kZXggd2l0aCB0d28gc2hhcmRzLiBEb2N1bWVudCAxIGxhbmRlZCBvbiBTaGFyZCAxIHdoaWxlIERvY3VtZW50cyAyIGFuZCAzIGxhbmRlZCBvbiBTaGFyZCAyLiBEb2N1bWVudCAxIGdvdCBhIGhpZ2hlciBzY29yZSBiZWNhdXNlIHdpdGhpbiBTaGFyZCAxLCBcdTIwMWNkb2dcdTIwMWQgaXMgYSByYXJlciB0ZXJtIFx1MjAxNCBpdCBvbmx5IG9jY3VycyBvbmNlLiBXaXRoaW4gU2hhcmQgMiwgXHUyMDFjZG9nXHUyMDFkIGlzIG1vcmUgY29tbW9uIFx1MjAxNCBpdCBvY2N1cnMgdHdpY2UuIEhlcmVcdTIwMTlzIGhvdyBJIHNldCB1cCB0aGUgZXhhbXBsZTo8XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwcmUgY2xhc3M9XCJ3cC1ibG9jay1wcmVmb3JtYXR0ZWRcIj48c3BhbiBjbGFzcz1cImhhcy1pbmxpbmUtY29sb3IgaGFzLXZpdmlkLWN5YW4tYmx1ZS1jb2xvclwiPlBVVCBcL3Rlc3QgXG57IFwic2V0dGluZ3NcIjogeyBcIm51bWJlcl9vZl9zaGFyZHNcIjogMiB9IH1cblxuUFVUIFwvdGVzdFwvX2RvY1wvMT9yb3V0aW5nPTBcbnsgXCJ0aXRsZVwiIDogXCJkb2dcIiB9IFxuXG5QVVQgXC90ZXN0XC9fZG9jXC8yP3JvdXRpbmc9MVxueyBcInRpdGxlXCIgOiBcImRvZ1wiIH0gXG5cblBVVCBcL3Rlc3RcL19kb2NcLzM\\\/cm91dGluZz0xXG57IFwidGl0bGVcIiA6IFwiZG9nXCIgfSA8XC9zcGFuPjxcL3ByZT5cbjxwPjwhLS0gXC93cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5JZiB5b3VcdTIwMTlyZSB3b3JraW5nIHdpdGggbXVsdGlwbGUgc2hhcmRzIGFuZCB5b3Ugd2FudCBzY29yZXMgdG8gYmUgY29uc2lzdGVudCByZWdhcmRsZXNzIG9mIHdoaWNoIHNoYXJkIGEgZG9jdW1lbnQgbGl2ZXMgaW4sIHlvdSBjYW4gZG8gYSBEaXN0cmlidXRlZCBGcmVxdWVuY3kgU2VhcmNoIGJ5IGFkZGVkIHRoZSBmb2xsb3dpbmcgcGFyYW1ldGVyIHRvIHlvdXIgcXVlcnk6XHUwMGEwPHN0cm9uZz5zZWFyY2hfdHlwZT1kZnNfcXVlcnlfdGhlbl9mZXRjaDxcL3N0cm9uZz4uIFRoaXMgdGVsbHMgRWxhc3RpY3NlYXJjaCB0byByZXRyaWV2ZSB0ZXJtIHN0YXRpc3RpY3MgZnJvbSBhbGwgdGhlIHNoYXJkcyBhbmQgY29tYmluZSB0aGVtIGJlZm9yZSBjb21wdXRpbmcgdGhlIHNjb3Jlcy48XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPkJ1dCBpdHMgYWxzbyBpbXBvcnRhbnQgdG8ga25vdyB0aGF0PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48ZmlndXJlIGNsYXNzPVwid3AtYmxvY2stcHVsbHF1b3RlXCI+XG48YmxvY2txdW90ZT5cbjxwPlJlcGxpY2FzIG9mIGEgc2hhcmQgbWF5IGhhdmUgZGlmZmVyZW50IHRlcm0gc3RhdGlzdGljcy48XC9wPlxuPFwvYmxvY2txdW90ZT5cbjxcL2ZpZ3VyZT5cbjxwPjwhLS0gXC93cDpwdWxscXVvdGUgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5UaGlzIGlzIGEgY29uc2VxdWVuY2Ugb2YgaG93IGRlbGV0aW9uIGlzIGhhbmRsZWQgaW4gTHVjZW5lLiBEb2N1bWVudHMgdGhhdCBhcmUgbWFya2VkIGZvciBkZWxldGlvbiBidXQgbm90IHlldCBwaHlzaWNhbGx5IHJlbW92ZWQgKHdoZW4gdGhlaXIgc2VnbWVudHMgYXJlIG1lcmdlZCkgc3RpbGwgY29udHJpYnV0ZSB0byB0ZXJtIHN0YXRpc3RpY3MuIFdoZW4gYSBkb2N1bWVudCBpcyBkZWxldGVkLCBhbGwgdGhlIHJlcGxpY2FzIHdpbGwgaW1tZWRpYXRlbHkgXHUyMDFja25vd1x1MjAxZCBhYm91dCB0aGUgZGVsZXRpb24sIGJ1dCB0aGV5IG1pZ2h0IG5vdCBjYXJyeSBpdCBvdXQgcGh5c2ljYWxseSBhdCB0aGUgc2FtZSB0aW1lLCBzbyB0aGV5IG1pZ2h0IGVuZCB1cCB3aXRoIGRpZmZlcmVudCB0ZXJtIHN0YXRpc3RpY3MuIFRvIHJlZHVjZSB0aGUgaW1wYWN0IG9mIHRoaXMsIHlvdSBjYW4gc3BlY2lmeSBhIHVzZXIgb3Igc2Vzc2lvbiBJRCBpbiB0aGUgc2hhcmQgY29weVx1MDBhMDxzdHJvbmc+cHJlZmVyZW5jZTxcL3N0cm9uZz5cdTAwYTBwYXJhbWV0ZXIuIFRoaXMgZW5jb3VyYWdlcyBFbGFzdGljc2VhcmNoIHRvIHJvdXRlIHJlcXVlc3RzIGZyb20gdGhlIHNhbWUgdXNlciB0byB0aGUgc2FtZSByZXBsaWNhcywgc28gdGhhdCwgZm9yIGV4YW1wbGUsIGEgdXNlciB3aWxsIG5vdCBub3RpY2Ugc2NvcmluZyBkaXNjcmVwYW5jaWVzIHdoZW4gaXNzdWluZyB0aGUgc2FtZSBxdWVyeSBtdWx0aXBsZSB0aW1lcy48XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPkl0cyBhbHNvIGltcG9ydGFudCB0byBrbm93IHRoYXQsIGZyb20gYSBzY29yaW5nIHBlcnNwZWN0aXZlLDxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnB1bGxxdW90ZSAtLT48XC9wPlxuPGZpZ3VyZSBjbGFzcz1cIndwLWJsb2NrLXB1bGxxdW90ZVwiPlxuPGJsb2NrcXVvdGU+XG48cD5Eb2N1bWVudCB1cGRhdGVzIGJlaGF2ZSBsaWtlIGluc2VydGlvbnMsIHVudGlsIHNlZ21lbnRzIGFyZSBtZXJnZWQuPFwvcD5cbjxcL2Jsb2NrcXVvdGU+XG48XC9maWd1cmU+XG48cD48IS0tIFwvd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+V2hlbiB5b3UgdXBkYXRlIGEgZG9jdW1lbnQgaW4gTHVjZW5lLCBhIG5ldyB2ZXJzaW9uIG9mIGl0IGlzIHdyaXR0ZW4gdG8gZGlzayBhbmQgdGhlIG9sZCB2ZXJzaW9uIGlzIG1hcmtlZCBmb3IgZGVsZXRpb24uIEJ1dCB0aGUgb2xkIHZlcnNpb24gY29udGludWVzIHRvIGNvbnRyaWJ1dGUgdG8gdGVybSBzdGF0aXN0aWNzIHVudGlsIGl0IGlzIHBoeXNpY2FsbHkgZGVsZXRlZC48XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPkluIHRoZSBleGFtcGxlIGJlbG93LCBJIGNyZWF0ZSBhIFx1MjAxY2RvZyBjYXRcdTIwMWQgZG9jdW1lbnQgYW5kIHRoZW4gSSB1cGRhdGUgaXRzIGNvbnRlbnRzIHRvIGJlIFx1MjAxY2RvZyB6ZWJyYS5cdTIwMWQgSW1tZWRpYXRlbHkgYWZ0ZXIgdGhlIHVwZGF0ZSwgaWYgSSBxdWVyeSBmb3IgXHUyMDFjZG9nXHUyMDFkIGFuZCBsb29rIGF0IHRoZSBleHBsYWluIG91dHB1dCwgRWxhc3RpY3NlYXJjaCB0ZWxscyBtZSB0aGVyZSBhcmVcdTAwYTA8ZW0+dHdvXHUwMGEwPFwvZW0+ZG9jdW1lbnRzIGNvbnRhaW5pbmcgdGhlIHRlcm0gXHUyMDFjZG9nLlx1MjAxZCBUaGUgbnVtYmVyIGdvZXMgZG93biB0byBvbmUgYWZ0ZXIgSSBkbyBhXHUwMGEwPHN0cm9uZz5fZm9yY2VtZXJnZTxcL3N0cm9uZz4uIFRoZSBtb3JhbDogaWYgeW91XHUyMDE5cmUgZG9pbmcgcmVsZXZhbmN5IHR1bmluZyBpbiBFbGFzdGljc2VhcmNoLCBsb29raW5nIGNsb3NlbHkgYXQgc2NvcmVzLCBhbmQgYWxzbyB1cGRhdGluZyBkb2N1bWVudHMgYXQgdGhlIHNhbWUgdGltZSwgYmUgc3VyZSB0byBydW4gYSBfZm9yY2VtZXJnZSBhZnRlciB5b3VyIHVwZGF0ZXMsIG9yIGVsc2UgcmVidWlsZCB0aGUgaW5kZXggZW50aXJlbHkuPFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5QVVQgdGVzdFwvX2RvY1wvMVxueyBcInRpdGxlXCI6IFwiZG9nIGNhdFwiIH1cblxuR0VUIHRlc3RcL19zZWFyY2g\\\/Zm9ybWF0PXlhbWxcbnsgXCJxdWVyeVwiIDogeyBcIm1hdGNoXCIgOiB7IFwidGl0bGVcIjogXCJkb2dcIiB9IH0sIFxuICBcImV4cGxhaW5cIjogdHJ1ZSB9XG5cblBVVCB0ZXN0XC9fZG9jXC8xP3JlZnJlc2hcbnsgXCJ0aXRsZVwiOiBcImRvZyB6ZWJyYVwiIH1cblxuR0VUIHRlc3RcL19zZWFyY2g\\\/Zm9ybWF0PXlhbWxcbnsgXCJxdWVyeVwiIDogeyBcIm1hdGNoXCIgOiB7IFwidGl0bGVcIjogXCJkb2dcIiB9IH0sIFxuICBcImV4cGxhaW5cIjogdHJ1ZSB9XG5cblBPU1QgdGVzdFwvX2ZvcmNlbWVyZ2VcblxuR0VUIHRlc3RcL19zZWFyY2g\\\/Zm9ybWF0PXlhbWxcbnsgXCJxdWVyeVwiIDogeyBcIm1hdGNoXCIgOiB7IFwidGl0bGVcIjogXCJkb2dcIiB9IH0sIFxuICBcdTIwMWNleHBsYWluXCI6IHRydWUgfTxcL3NwYW4+PFwvcHJlPlxuPHA+PCEtLSBcL3dwOnByZWZvcm1hdHRlZCAtLT48XC9wPlxuPHA+PCEtLSB3cDpoZWFkaW5nIHtcImxldmVsXCI6NH0gLS0+PFwvcD5cbjxoND5RdWVyeSA4OiBkb2cgY2F0PFwvaDQ+XG48cD48IS0tIFwvd3A6aGVhZGluZyAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPk5vdyBsZXRcdTIwMTlzIHRha2UgYW5vdGhlciBsb29rIGF0IFF1ZXJ5IDQsIHdoZXJlIHdlIHNlYXJjaGVkIGZvciBcdTIwMWNkb2cgY2F0XHUyMDFkIGFuZCB3ZSBmb3VuZCB0aGF0IGEgZG9jdW1lbnQgY29udGFpbmluZyBib3RoIHRlcm1zIGRpZCBiZXR0ZXIgdGhhbiBkb2N1bWVudHMgd2l0aCBsb3RzIG9mIGluc3RhbmNlcyBvZiBvbmUgdGVybSBvciB0aGUgb3RoZXIuIEluIFF1ZXJ5IDQsIHdlIHdlcmUgc2VhcmNoaW5nIHRoZSB0aXRsZSBmaWVsZCwgdGhlIG9ubHkgZmllbGQgYXZhaWxhYmxlLiBIZXJlLCB3ZVx1MjAxOXZlIGdvdFx1MDBhMDxlbT50d288XC9lbT5cdTAwYTBmaWVsZHMsIHBldDEgYW5kIHBldDI6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5Eb2MgMToge1wicGV0MVwiOiBcImRvZ1wiLCBcInBldDJcIjogXCJkb2dcIn1cbkRvYyAyOiB7XCJwZXQxXCI6IFwiZG9nXCIsIFwicGV0MlwiOiBcImNhdFwifTxcL3NwYW4+PFwvcHJlPlxuPHA+PCEtLSBcL3dwOnByZWZvcm1hdHRlZCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPldlXHUyMDE5bGwgZG8gYSBtdWx0aV9tYXRjaCBpbmNsdWRpbmcgdGhvc2UgdHdvIGZpZWxkcyBsaWtlIHRoaXM6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5HRVQgdGVzdFwvX3NlYXJjaFxue1xuICBcInF1ZXJ5XCI6IHtcbiAgIFwibXVsdGlfbWF0Y2hcIjoge1xuICAgIFwicXVlcnlcIjogXCJkb2cgY2F0XCIsIFxuICAgIFwiZmllbGRzXCI6IFtcdTIwMWNwZXQxXHUyMDFkLCBcInBldDJcIl0sXG4gICAgXCJ0eXBlXCI6IFwibW9zdF9maWVsZHNcIlxuICB9XG4gfVxufTxcL3NwYW4+PFwvcHJlPlxuPHA+PCEtLSBcL3dwOnByZWZvcm1hdHRlZCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPlNpbmNlIERvY3VtZW50IDIgbWF0Y2hlcyBib3RoIG9mIG91ciBxdWVyeSB0ZXJtcyB3ZVx1MjAxOWQgY2VydGFpbmx5IGhvcGUgdGhhdCBpdCBkb2VzIGJldHRlciB0aGFuIERvY3VtZW50IDEuIFRoYXRcdTIwMTlzIHRoZSBsZXNzb24gd2UgdG9vayBhd2F5IGZyb20gUXVlcnkgNCwgcmlnaHQ\\\/IFdlbGwsIGhlcmUgYXJlIHRoZSByZXN1bHRzOjxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnRhYmxlIHtcImhhc0ZpeGVkTGF5b3V0XCI6dHJ1ZX0gLS0+PFwvcD5cbjxmaWd1cmUgY2xhc3M9XCJ3cC1ibG9jay10YWJsZVwiPlxuPHRhYmxlIGNsYXNzPVwiaGFzLWZpeGVkLWxheW91dFwiPlxuPHRib2R5PlxuPHRyPlxuPHRkPjxzdHJvbmc+SUQ8XC9zdHJvbmc+PFwvdGQ+XG48dGQ+PHN0cm9uZz5QZXQgMTxcL3N0cm9uZz48XC90ZD5cbjx0ZD48c3Ryb25nPlBldCAyPFwvc3Ryb25nPjxcL3RkPlxuPHRkPjxzdHJvbmc+U2NvcmU8XC9zdHJvbmc+PFwvdGQ+XG48XC90cj5cbjx0cj5cbjx0ZD4xPFwvdGQ+XG48dGQ+ZG9nPFwvdGQ+XG48dGQ+ZG9nPFwvdGQ+XG48dGQ+MC44NzxcL3RkPlxuPFwvdHI+XG48dHI+XG48dGQ+MjxcL3RkPlxuPHRkPmRvZzxcL3RkPlxuPHRkPmNhdDxcL3RkPlxuPHRkPjAuODc8XC90ZD5cbjxcL3RyPlxuPFwvdGJvZHk+XG48XC90YWJsZT5cbjxcL2ZpZ3VyZT5cbjxwPjwhLS0gXC93cDp0YWJsZSAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPllvdSBjYW4gc2VlIHRoZSBkb2N1bWVudHMgYXJlIHRpZWQuIEl0IG1pZ2h0IGxvb2sgbGlrZSBcdTIwMWNjYXRcdTIwMWQgaXMgYSByYXJlIHRlcm0gdGhhdCBzaG91bGQgaGVscCBEb2N1bWVudCAyIHJpc2UgdG8gdGhlIHRvcCwgYnV0IHdpdGhpbiB0aGUgUGV0IDIgZmllbGQsIFx1MjAxY2NhdFx1MjAxZCBhbmQgXHUyMDFjZG9nXHUyMDFkIGhhdmUgdGhlXHUwMGEwPGVtPnNhbWU8XC9lbT5cdTAwYTBkb2N1bWVudCBmcmVxdWVuY2llcywgYW5kIHRoZSBzY29yaW5nIGlzIGRvbmUgb24gYSBwZXItZmllbGQgYmFzaXMuIEl0IGFsc28gbG9va3MgbGlrZSBEb2N1bWVudCAyIHNob3VsZCBnZXQgYW4gYWR2YW50YWdlIGZvciBtYXRjaGluZ1x1MDBhMDxlbT5tb3JlPFwvZW0+XHUwMGEwb2YgdGhlIHF1ZXJ5IHRlcm1zLCBidXQgYWdhaW4sIHRoZSBzY29yaW5nIGlzIGRvbmUgYSBwZXItZmllbGQgYmFzaXM6IHdoZW4gd2UgY29tcHV0ZSB0aGUgc2NvcmUgaW4gdGhlIFBldCAxIGZpZWxkLCBib3RoIGRvY3VtZW50cyBkbyB0aGUgc2FtZTsgd2hlbiB3ZSBjb21wdXRlIHRoZSBzY29yZSBpbiB0aGUgUGV0IDIgZmllbGQsIGJvdGggZG9jdW1lbnRzIGFnYWluIGRvIHRoZSBzYW1lLjxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+RG9lcyB0aGlzIGNvbnRyYWRpY3Qgd2hhdCB3ZSBsZWFybmVkIGZyb20gUXVlcnkgND8gTm90IHF1aXRlLCBidXQgaXQgd2FycmFudHMgYSByZWZpbmVtZW50IG9mIHRoZSBlYXJsaWVyIGxlc3Nvbjo8XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwdWxscXVvdGUgLS0+PFwvcD5cbjxmaWd1cmUgY2xhc3M9XCJ3cC1ibG9jay1wdWxscXVvdGVcIj5cbjxibG9ja3F1b3RlPlxuPHA+TWF0Y2hpbmcgbW9yZSBxdWVyeSB0ZXJtcyB3aXRoaW4gdGhlIHNhbWUgZmllbGQgaXMgZ29vZC4gQnV0IHRoZXJlXHUyMDE5cyBubyBhZHZhbnRhZ2Ugd2hlbiB0aGUgbWF0Y2hlcyBoYXBwZW4gYWNyb3NzIGZpZWxkcy48XC9wPlxuPFwvYmxvY2txdW90ZT5cbjxcL2ZpZ3VyZT5cbjxwPjwhLS0gXC93cDpwdWxscXVvdGUgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5JZiB5b3VcdTIwMTlyZSBub3QgaGFwcHkgd2l0aCB0aGlzIHNpdHVhdGlvbiwgdGhlcmUgYXJlIHNvbWUgdGhpbmdzIHlvdSBjYW4gZG8uIFlvdSBjYW4gY29tYmluZSB0aGUgY29udGVudHMgb2YgUGV0IDEgYW5kIFBldCAyIGludG8gYSBzaW5nbGUgZmllbGQuIFlvdSBjYW4gYWxzbyBzd2l0Y2ggZnJvbSB0aGVcdTAwYTA8c3Ryb25nPm1vc3RfZmllbGRzPFwvc3Ryb25nPlx1MDBhMHRvIHRoZVx1MDBhMDxzdHJvbmc+Y3Jvc3NfZmllbGRzPFwvc3Ryb25nPlx1MDBhMHF1ZXJ5IHR5cGUgdG8gc2ltdWxhdGUgYSBzaW5nbGUgZmllbGQuIChKdXN0IGJlIGF3YXJlIHRoYXQgY3Jvc3NfZmllbGRzIGhhcyBzb21lIG90aGVyIGNvbnNlcXVlbmNlcyBvbiBzY29yaW5nLCBjaGFuZ2luZyB0aGUgcHJvY2VkdXJlIGZyb20gZmllbGQtY2VudHJpYyB0byB0ZXJtLWNlbnRyaWMuIFdlIHdvblx1MjAxOXQgZ28gaW50byBkZXRhaWxzIGhlcmUuKTxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOmhlYWRpbmcge1wibGV2ZWxcIjo0fSAtLT48XC9wPlxuPGg0PlF1ZXJ5IDk6IG9yYW5nZSBkb2c8XC9oND5cbjxwPjwhLS0gXC93cDpoZWFkaW5nIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+Tm93IGxldFx1MjAxOXMgcmV2aXNpdCB0aGUgbGVzc29uIGZyb20gUXVlcnkgNSwgd2hlcmUgd2Ugc2F3IHRoYXQgbWF0Y2hlcyBpbiBzaG9ydGVyIGZpZWxkcyBhcmUgYmV0dGVyLiBXZVx1MjAxOXJlIGdvaW5nIHRvIHNlYXJjaCBmb3IgXHUyMDFjb3JhbmdlIGRvZy5cdTIwMWQgV2UgaGF2ZSBhIFx1MjAxY2RvZ1x1MjAxZCBkb2N1bWVudCB3aXRoIGEgZGVzY3JpcHRpb24gbWVudGlvbmluZyB0aGF0IHRoZSBkb2cgaXMgYnJvd24uIEFuZCB3ZSBoYXZlIGEgXHUyMDFjY2F0XHUyMDFkIGRvY3VtZW50IHdpdGggYSBkZXNjcmlwdGlvbiBtZW50aW9uaW5nIHRoYXQgdGhlIGNhdCBpcyBzb21ldGltZXMgb3JhbmdlLiBOb3RpY2UgdGhhdCB0aGUgZG9nIGRvY3VtZW50IGhhcyBhIGxvbmdlciBkZXNjcmlwdGlvbiB0aGFuIHRoZSBjYXQgZG9jdW1lbnQsIGFuZCBib3RoIGRlc2NyaXB0aW9ucyBhcmUgbG9uZ2VyIHRoYW4gdGhlIGNvbnRlbnRzIG9mIHRoZSB0eXBlIGZpZWxkLjxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnByZWZvcm1hdHRlZCAtLT48XC9wPlxuPHByZSBjbGFzcz1cIndwLWJsb2NrLXByZWZvcm1hdHRlZFwiPjxzcGFuIGNsYXNzPVwiaGFzLWlubGluZS1jb2xvciBoYXMtdml2aWQtY3lhbi1ibHVlLWNvbG9yXCI+RG9jIDE6IHtcdTIwMWN0eXBlXHUyMDFkOiBcdTIwMWNkb2dcdTIwMWQsIFx1MjAxY2Rlc2NyaXB0aW9uXHUyMDFkOiBcdTIwMWNBIHN3ZWV0IGFuZCBsb3ZpbmcgcGV0IHRoYXQgaXMgYWx3YXlzIGVhZ2VyIHRvIHBsYXkuIEJyb3duIGNvYXQuIExvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0LCBjb25zZWN0ZXR1ciBhZGlwaXNjaW5nIGVsaXQuIER1aXMgbm9uIG5pYmggc2FnaXR0aXMsIG1vbGxpcyBleCBhLCBzY2VsZXJpc3F1ZSBuaXNsLiBVdCB2aXRhZSBwZWxsZW50ZXNxdWUgbWFnbmEsIHV0IHRyaXN0aXF1ZSBuaXNpLiBNYWVjZW5hcyB1dCB1cm5hIGEgZWxpdCBwb3N1ZXJlIHNjZWxlcmlzcXVlLiBTdXNwZW5kaXNzZSB2ZWwgdXJuYSB0dXJwaXMuIE1hdXJpcyB2aXZlcnJhIGZlcm1lbnR1bSB1bGxhbWNvcnBlci4gRHVpcyBhYyBsYWN1cyBuaWJoLiBOdWxsYSBhdWN0b3IgbGFjdXMgaW4gcHVydXMgdnVscHV0YXRlLCBtYXhpbXVzIHVsdHJpY2llcyBhdWd1ZSBzY2VsZXJpc3F1ZS5cdTIwMWR9XG5cbkRvYyAyOiB7XHUyMDFjdHlwZVx1MjAxZDogXHUyMDFjY2F0XHUyMDFkLCBcdTIwMWNkZXNjcmlwdGlvblx1MjAxZDogXHUyMDFjUHV6emxpbmdseSBncnVtcHkuIE9jY2FzaW9uYWxseSB0dXJucyBvcmFuZ2UuXHUyMDFkfTxcL3NwYW4+PFwvcHJlPlxuPHA+PCEtLSBcL3dwOnByZWZvcm1hdHRlZCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPldlXHUyMDE5bGwgZG8gYSBtdWx0aV9tYXRjaCBsaWtlIHRoaXM6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHJlZm9ybWF0dGVkIC0tPjxcL3A+XG48cHJlIGNsYXNzPVwid3AtYmxvY2stcHJlZm9ybWF0dGVkXCI+PHNwYW4gY2xhc3M9XCJoYXMtaW5saW5lLWNvbG9yIGhhcy12aXZpZC1jeWFuLWJsdWUtY29sb3JcIj5HRVQgdGVzdFwvX3NlYXJjaFxue1xuICBcInF1ZXJ5XCI6IHtcbiAgIFwibXVsdGlfbWF0Y2hcIjoge1xuICAgIFwicXVlcnlcIjogXCJvcmFuZ2UgZG9nXCIsIFxuICAgIFwiZmllbGRzXCI6IFtcInR5cGVcIiwgXHUyMDFjZGVzY3JpcHRpb25cIl0sXG4gICAgXCJ0eXBlXCI6IFwibW9zdF9maWVsZHNcIlxuICB9XG4gfVxufTxcL3NwYW4+PFwvcHJlPlxuPHA+PCEtLSBcL3dwOnByZWZvcm1hdHRlZCAtLT48XC9wPlxuPHA+PCEtLSB3cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPldoYXRcdTIwMTlzIGdvaW5nIHRvIHRha2UgcHJlY2VkZW5jZSBoZXJlOiB0aGUgbWF0Y2ggZm9yIFx1MjAxY2RvZ1x1MjAxZCBpbiB0aGUgdHlwZSBmaWVsZCwgd2hpY2ggaXMgYSByZWFsbHkgc2hvcnQgZmllbGQsIG9yIHRoZSBtYXRjaCBmb3IgXHUyMDFjb3JhbmdlXHUyMDFkIGluIHRoZSBkZXNjcmlwdGlvbiBmaWVsZCwgd2hpY2ggaXMgc2lnbmlmaWNhbnRseSBsb25nZXI\\\/IElmIG1hdGNoZXMgaW4gc2hvcnRlciBmaWVsZHMgYXJlIGJldHRlciwgc2hvdWxkblx1MjAxOXQgXHUyMDFjZG9nXHUyMDFkIHdpbiBoZXJlPyBJbiBmYWN0LCB0aGUgcmVzdWx0cyBsb29rIGxpa2UgdGhpczo8XC9wPlxuPHA+PCEtLSBcL3dwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+PCEtLSB3cDp0YWJsZSB7XCJoYXNGaXhlZExheW91dFwiOnRydWV9IC0tPjxcL3A+XG48ZmlndXJlIGNsYXNzPVwid3AtYmxvY2stdGFibGVcIj5cbjx0YWJsZSBjbGFzcz1cImhhcy1maXhlZC1sYXlvdXRcIj5cbjx0Ym9keT5cbjx0cj5cbjx0ZD5JRDxcL3RkPlxuPHRkPjxzdHJvbmc+VHlwZTxcL3N0cm9uZz48XC90ZD5cbjx0ZD48c3Ryb25nPkRlc2NyaXB0aW9uPFwvc3Ryb25nPjxcL3RkPlxuPHRkPjxzdHJvbmc+U2NvcmU8XC9zdHJvbmc+PFwvdGQ+XG48XC90cj5cbjx0cj5cbjx0ZD4xPFwvdGQ+XG48dGQ+ZG9nPFwvdGQ+XG48dGQ+QSBzd2VldFx1MjAyNiBicm93blx1MjAyNjxcL3RkPlxuPHRkPjAuNjk8XC90ZD5cbjxcL3RyPlxuPHRyPlxuPHRkPjI8XC90ZD5cbjx0ZD5jYXQ8XC90ZD5cbjx0ZD5QdXp6bGluZ2x5IGdydW1weVx1MjAyNiBvcmFuZ2UuPFwvdGQ+XG48dGQ+MS4wNjxcL3RkPlxuPFwvdHI+XG48XC90Ym9keT5cbjxcL3RhYmxlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnRhYmxlIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+VGhlIG1hdGNoIGZvciBcdTIwMWNvcmFuZ2VcdTIwMWQgaW4gdGhlIERlc2NyaXB0aW9uIGZpZWxkIGlzIHNlbmRpbmcgRG9jdW1lbnQgMiB0byB0aGUgdG9wIGV2ZW4gdGhvdWdoIHRoYXQgbWF0Y2ggb2NjdXJzIGluIGEgbG9uZ2VyIGZpZWxkIGJvZHkgdGhhbiB0aGUgbWF0Y2ggZm9yIFx1MjAxY2RvZ1x1MjAxZCBpbiBEb2N1bWVudCAxLiBEb2VzIHRoaXMgY29udHJhZGljdCB3aGF0IHdlIGxlYXJuZWQgYWJvdXQgbWF0Y2hlcyBpbiBzaG9ydCBhbmQgbG9uZyBmaWVsZHMgZnJvbSBRdWVyeSA1PyBObywgYnV0IGl0IHBvaW50cyB0byBzb21ldGhpbmcgd2UgaGFkblx1MjAxOXQgbWVudGlvbmVkLiBUaGUgbGVzc29uIGlzIHRoYXQ6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48ZmlndXJlIGNsYXNzPVwid3AtYmxvY2stcHVsbHF1b3RlXCI+XG48YmxvY2txdW90ZT5cbjxwPlx1MjAxY1Nob3J0bmVzc1x1MjAxZCBpcyByZWxhdGl2ZSB0byB0aGUgZmllbGRcdTIwMTlzIGF2ZXJhZ2UuPFwvcD5cbjxcL2Jsb2NrcXVvdGU+XG48XC9maWd1cmU+XG48cD48IS0tIFwvd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+V2l0aGluIHRoZSB0eXBlIGZpZWxkLCBcdTIwMWNkb2dcdTIwMWQgYW5kIFx1MjAxY2NhdFx1MjAxZCBkb25cdTIwMTl0IGdldCBhbiBhZHZhbnRhZ2UgZm9yIGJlaW5nIHNob3J0IGJlY2F1c2UsIGluIGZhY3QsIHRoZXlcdTIwMTlyZSBib3RoIG9mIGF2ZXJhZ2UgbGVuZ3RoIGZvciB0aGF0IGZpZWxkLiBPbiB0aGUgb3RoZXIgaGFuZCwgdGhlIGRlc2NyaXB0aW9uIGZvciB0aGUgY2F0IGlzIHNob3J0ZXIgdGhhbiBhdmVyYWdlIGZvciB0aGUgZGVzY3JpcHRpb24gZmllbGQgb3ZlcmFsbCwgc28gaXQgZ2V0cyBhIGJlbmVmaXQgZm9yIGJlaW5nIFx1MjAxY3Nob3J0Llx1MjAxZDxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOmhlYWRpbmcge1wibGV2ZWxcIjo0fSAtLT48XC9wPlxuPGg0PlF1ZXJ5IDEwOiBhYmNkIGVmZ2hpamtsbW5vcHFyc3R1dnd4eXo8XC9oND5cbjxwPjwhLS0gXC93cDpoZWFkaW5nIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+SGVyZVx1MjAxOXMgYW4gZWFzeSBvbmUgdG8gY2xvc2Ugb3V0IG91ciBzZXQgb2YgZXhhbXBsZXMuIElcdTIwMTl2ZSBkaXZpZGVkIHRoZSBhbHBoYWJldCBpbnRvIHR3byBxdWVyeSB0ZXJtcy4gT25lIG9mIHRoZW0gaXMgYSBzaG9ydCB0ZXJtLCBcdTIwMWNhYmNkLFx1MjAxZCBhbmQgdGhlIG90aGVyIGlzIGEgbG9uZyB0ZXJtLCBcdTIwMWNlZmdoaWprbG1ub3BxcnN0dXZ3eHl6Llx1MjAxZCBJXHUyMDE5bSBnb2luZyB0byBzZWFyY2ggZm9yIGJvdGggdGVybXMgdG9nZXRoZXI6IFx1MjAxY2FiY2QgZWZnaGlqa2xtbm9wcXJzdHV2d3h5ei5cdTIwMWQgTXkgaW5kZXggaGFzIG9uZSBtYXRjaCBmb3IgZWFjaCB0ZXJtOjxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnByZWZvcm1hdHRlZCAtLT48XC9wPlxuPHByZSBjbGFzcz1cIndwLWJsb2NrLXByZWZvcm1hdHRlZFwiPjxzcGFuIGNsYXNzPVwiaGFzLWlubGluZS1jb2xvciBoYXMtdml2aWQtY3lhbi1ibHVlLWNvbG9yXCI+RG9jIDE6IFwiYWJjZFwiXG5Eb2MgMjogXCJlZmdoaWprbG1ub3BxcnN0dXZ3eHl6XCI8XC9zcGFuPjxcL3ByZT5cbjxwPjwhLS0gXC93cDpwcmVmb3JtYXR0ZWQgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5XaGljaCBkb2N1bWVudCBpcyBnb2luZyB0byBkbyB0aGUgYmVzdD8gVGhlIHJlc3VsdHMgbG9vayBsaWtlIHRoaXM6PFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6dGFibGUge1wiaGFzRml4ZWRMYXlvdXRcIjp0cnVlfSAtLT48XC9wPlxuPGZpZ3VyZSBjbGFzcz1cIndwLWJsb2NrLXRhYmxlXCI+XG48dGFibGUgY2xhc3M9XCJoYXMtZml4ZWQtbGF5b3V0XCI+XG48dGJvZHk+XG48dHI+XG48dGQ+SUQ8XC90ZD5cbjx0ZD48c3Ryb25nPlRpdGxlPFwvc3Ryb25nPjxcL3RkPlxuPHRkPjxzdHJvbmc+U2NvcmU8XC9zdHJvbmc+PFwvdGQ+XG48XC90cj5cbjx0cj5cbjx0ZD4xPFwvdGQ+XG48dGQ+YWJjZDxcL3RkPlxuPHRkPjAuNjk8XC90ZD5cbjxcL3RyPlxuPHRyPlxuPHRkPjI8XC90ZD5cbjx0ZD5lZmdoaWprbG1ub3BxcnN0dXZ3eHl6PFwvdGQ+XG48dGQ+MC42OTxcL3RkPlxuPFwvdHI+XG48XC90Ym9keT5cbjxcL3RhYmxlPlxuPFwvZmlndXJlPlxuPHA+PCEtLSBcL3dwOnRhYmxlIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+V2h5IGFyZSB0aGUgZG9jdW1lbnRzIHRpZWQgaWYgaXRcdTIwMTlzIHRydWUgdGhhdCBtYXRjaGVzIGluIHNob3J0ZXIgZmllbGRzIGFyZSBiZXR0ZXI\\\/IFRoZSBsZXNzb24gaXMgdGhhdDxcL3A+XG48cD48IS0tIFwvd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD48IS0tIHdwOnBhcmFncmFwaCAtLT48XC9wPlxuPHA+V2hlbiB3ZSB0YWxrIGFib3V0IGEgc2hvcnQgb3IgbG9uZyBmaWVsZCwgd2VcdTIwMTlyZSB0YWxraW5nIGFib3V0IGhvdyBtYW55XHUwMGEwPGVtPnRlcm1zPFwvZW0+XHUwMGEwdGhlIGZpZWxkIGNvbnRhaW5zLCBub3QgaG93IG1hbnlcdTAwYTA8ZW0+Y2hhcmFjdGVyczxcL2VtPi4gSW4gdGhpcyBleGFtcGxlLCB0aGUgdGl0bGVzIGZvciBib3RoIGRvY3VtZW50cyBhcmUgb2YgbGVuZ3RoIDEuPFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD5cbjxwPjwhLS0gd3A6cHVsbHF1b3RlIC0tPjxcL3A+XG48ZmlndXJlIGNsYXNzPVwid3AtYmxvY2stcHVsbHF1b3RlXCI+XG48YmxvY2txdW90ZT5cbjxwPlRlcm0gbGVuZ3RoIGlzIG5vdCBzaWduaWZpY2FudC48XC9wPlxuPFwvYmxvY2txdW90ZT5cbjxcL2ZpZ3VyZT5cbjxwPjwhLS0gXC93cDpwdWxscXVvdGUgLS0+PFwvcD5cbjxwPjwhLS0gd3A6cGFyYWdyYXBoIC0tPjxcL3A+XG48cD5UaGF0XHUyMDE5cyBpdCBmb3Igbm93LiBIb3BlZnVsbHkgdGhlc2UgZXhhbXBsZXMgaGF2ZSBoZWxwZWQgYnVpbGQgeW91ciBpbnR1aXRpb25zIGFib3V0IGhvdyBzY29yaW5nIHdvcmtzIGluIEVsYXN0aWNzZWFyY2ggYW5kIFNvbHIuIFRoYW5rcyBmb3IgZm9sbG93aW5nIGFsb25nISBJZiB5b3VcdTIwMTlkIGxpa2UgdG8gZ28gZnVydGhlciBhbmQgdW5kZXJzdGFuZCBob3cgdGhlIEJNMjUgc2NvcmluZyBmdW5jdGlvbiBhY3R1YWxseSBhY2hpZXZlcyB0aGUgYmVoYXZpb3JzIHdlXHUyMDE5dmUgc2VlbiBoZXJlLCBjaGVjayBvdXQgb3VyIGNvbXBhbmlvbiBhcnRpY2xlIG9uXHUwMGEwPGEgaHJlZj1cImh0dHBzOlwvXC9rbXdsbGMuY29tXC9pbmRleC5waHBcLzIwMjBcLzAyXC8yNlwvdW5kZXJzdGFuZGluZy10Zi1pZGYtYW5kLWJtMjVcL1wiPlVuZGVyc3RhbmRpbmcgVEYtSURGIGFuZCBCTTI1PFwvYT4uPFwvcD5cbjxwPjwhLS0gXC93cDpwYXJhZ3JhcGggLS0+PFwvcD4ifSwiZWxlbWVudHMiOltdLCJ3aWRnZXRUeXBlIjoidGV4dC1lZGl0b3IifQ==\\\"]\\t\\t\\t<\\\/div>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t<\\\/div><\\\/div>\\r\\n\\t\\t<\\\/section>\\r\\n\\t\\t\\t\\t
\\r\\n\\t\\t\\t\\t\\t\\t
\\r\\n\\t\\t\\t\\t\\t
\\n\\t\\t\\t
\\n\\t\\t\\t\\t[elementor-element k=\\\"9109a976d8649ee6d2c8fef8daebbb8b\\\" data=\\\"eyJpZCI6IjA2NGMzODYiLCJlbFR5cGUiOiJ3aWRnZXQiLCJzZXR0aW5ncyI6eyJwcmV2X2xhYmVsIjoiUHJldmlvdXMgUG9zdCIsIm5leHRfbGFiZWwiOiJOZXh0IFBvc3QiLCJzaG93X2JvcmRlcnMiOiIiLCJ0aXRsZV90eXBvZ3JhcGh5X3R5cG9ncmFwaHkiOiJjdXN0b20iLCJ0aXRsZV90eXBvZ3JhcGh5X2ZvbnRfc2l6ZSI6eyJ1bml0IjoicHgiLCJzaXplIjoxNCwic2l6ZXMiOltdfSwidGl0bGVfdHlwb2dyYXBoeV9mb250X3dlaWdodCI6IjcwMCIsIl9fZ2xvYmFsc19fIjp7ImFycm93X2NvbG9yIjoiZ2xvYmFsc1wvY29sb3JzP2lkPXByaW1hcnkiLCJsYWJlbF9jb2xvciI6Imdsb2JhbHNcL2NvbG9ycz9pZD1zZWNvbmRhcnkifSwiYXJyb3ciOiJmYSBmYS1jYXJldC1sZWZ0Iiwic2hvd19hcnJvdyI6IiJ9LCJlbGVtZW50cyI6W10sIndpZGdldFR5cGUiOiJnbG9iYWwiLCJ0ZW1wbGF0ZUlEIjoiMjgwODMifQ==\\\"]\\t\\t\\t<\\\/div>\\n\\t\\t<\\\/div>\\n\\t\\t\\t\\t\\t<\\\/div><\\\/div>\\r\\n\\t\\t<\\\/section>\\r\\n\\t\\t\",\"scripts\":[],\"styles\":[]}}"]},"jetpack_featured_media_url":"https:\/\/kmwllc.com\/wp-content\/uploads\/2020\/03\/blog_ScoringExamples_1200x900.png","menu_order":0,"_links":{"self":[{"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/posts\/26175","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/comments?post=26175"}],"version-history":[{"count":10,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/posts\/26175\/revisions"}],"predecessor-version":[{"id":29727,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/posts\/26175\/revisions\/29727"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/media\/29724"}],"wp:attachment":[{"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/media?parent=26175"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/categories?post=26175"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kmwllc.com\/index.php\/wp-json\/wp\/v2\/tags?post=26175"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}