Extracting data from Web Pages

{
  "@context": "https://schema.org",
  "@type": "WebPage",
  "name": [
    "GitHub - Turnerj/Quickenshtein: Making the quickest and most memory efficient implementation of Levenshtein Distance with SIMD and Threading support",
    "Turnerj/Quickenshtein"
  ],
  "description": "Making the quickest and most memory efficient implementation of Levenshtein Distance with SIMD and Threading support - Turnerj/Quickenshtein",
  "alternateName": "GitHub",
  "image": {
    "@context": "https://schema.org",
    "@type": "ImageObject",
    "description": "Making the quickest and most memory efficient implementation of Levenshtein Distance with SIMD and Threading support - Turnerj/Quickenshtein",
    "url": "https://opengraph.githubassets.com/1047a70c92a880ac232cbb029197c4b09df13bf921fa39275159f72e1a544e3c/Turnerj/Quickenshtein",
    "height": 600,
    "width": 1200
  },
  "url": "https://github.com/Turnerj/Quickenshtein",
  "headline": "Turnerj/Quickenshtein",
  "mainEntity": {
    "@context": "https://schema.org",
    "@type": "SoftwareSourceCode",
    "name": "<a href=\"https://github.com/Turnerj/Quickenshtein\">Quickenshtein</a>",
    "license": "/Turnerj/Quickenshtein/blob/main/License.txt",
    "text": "<h1>Quickenshtein</h1> <p>A quick and memory efficient Levenshtein Distance calculator for .NET</p> <p><a href=\"https://ci.appveyor.com/project/Turnerj/Quickenshtein\"><img src=\"https://camo.githubusercontent.com/f8bd19156a94ff534be47484679d394b1b4bd297a9b424cdf295e05ec9d64845/68747470733a2f2f696d672e736869656c64732e696f2f6170707665796f722f63692f5475726e65726a2f517569636b656e73687465696e2f6d61737465722e737667\" alt=\"AppVeyor\"></a> <a href=\"https://codecov.io/gh/Turnerj/Quickenshtein\"><img src=\"https://camo.githubusercontent.com/f3d33f414d0ed126f1292d63e5a0b0f813cef58a7c1af07be68b5a714a1db3c9/68747470733a2f2f696d672e736869656c64732e696f2f636f6465636f762f632f6769746875622f5475726e65726a2f517569636b656e73687465696e2f6d61737465722e737667\" alt=\"Codecov\"></a> <a href=\"https://www.nuget.org/packages/Quickenshtein/\"><img src=\"https://camo.githubusercontent.com/a060bbf3ad717682e4e2ec0c8f5224fc078dc46f21bbe419c1691933d385d5df/68747470733a2f2f696d672e736869656c64732e696f2f6e756765742f762f517569636b656e73687465696e2e737667\" alt=\"NuGet\"></a></p> <h2>Performance</h2> <p>Quickenshtein gets its speed and memory efficiency from a number of different optimizations. To get the most performance out of the library, you will need .NET Core 3 or higher as this has support for hardware intrinsics.</p> <p>Quickenshtein takes advantage of the following hardware intrinsics. On any recent x86 system, you will likely have these available.</p> <ul><li><a href=\"https://en.wikipedia.org/wiki/SSE2#CPU_support\">SSE2</a></li><li><a href=\"https://en.wikipedia.org/wiki/SSE4#Supporting_CPUs\">SSE4.1</a></li><li><a href=\"https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2\">AVX2</a></li></ul> <p>If your computer doesn't have one of the hardware intrinsics available, Quickenshtein will still work - just slower than optimal.</p> <h2>Multi-Threading</h2> <p>By default, Quickenshtein performs in single-threaded mode as this mode performs best for small to medium size strings while having no memory allocations. When dealing with huge strings of 8000 characters or more, it may be useful to switch to multi-threaded mode. In this mode, the calculation is broken up and shared between multiple cores in a system.</p> <p>Multi-threading is especially useful for systems without hardware intrinsics or for .NET Framework as shown in the table below where it provided a 3x performance improvement.</p>  <p>To enable threading, you can pass in <code>CalculationOptions.DefaultWithThreading</code> to <code>Levenshtein.GetDistance()</code> or configure your own <code>CalculationOptions</code> with settings that work best for you.</p> <p><em>Note: Multi-threading is not allocation free (unlike single-threading mode) and will allocate a small amount depending on the number of threads used.</em></p> <h2>Benchmarking</h2> <p>There are a number of benchmarks in the repository that you can run on your system to see how well Quickenshtein performs.</p> <p>Most of these benchmarks...</p> <ul><li>Run .NET Framework and .NET Core so you can see how the performance changes between them</li><li>Compare against a simple baseline Levenshtein Distance implementation with no specific optimizations</li><li>Compare against <a href=\"https://github.com/DanHarltey/Fastenshtein/\">Fastenshtein</a>, one of the other fast .NET Levenshtein Distance implementations</li></ul> <p>You can view results to these benchmarks at the links below:</p> <ul><li><a href=\"https://github.com/Turnerj/Quickenshtein/blob/main/docs/OverallComparison.md\">Benchmarking against Fastenshtein</a></li><li><a href=\"https://github.com/Turnerj/Quickenshtein/blob/main/docs/StringScenarios.md\">Isolated Benchmarks</a></li><li><a href=\"https://github.com/Turnerj/Quickenshtein/blob/main/docs/IntrinsicsPerformance.md\">Intrinsics Performance</a></li></ul> <h2>Example Usage</h2> <pre>using Quickenshtein;\n\n// Common usage (uses default CalculationOptions with threading disabled)\nvar distance1 = Levenshtein.GetDistance(\"Saturday\", \"Sunday\");\n\n// Enable threading\nvar distance2 = Levenshtein.GetDistance(\"Saturday\", \"Sunday\", CalculationOptions.DefaultWithThreading);\n\n// Custom calculation options (helps with tuning for your specific workload and environment - you should benchmark your configurations on your system)\nvar distance3 = Levenshtein.GetDistance(\"Saturday\", \"Sunday\", new CalculationOptions {\n    EnableThreadingAfterXCharacters = 10000,\n    MinimumCharactersPerThread = 25000\n});</pre>"
  },
  "text": "<h1>Quickenshtein</h1> <p>A quick and memory efficient Levenshtein Distance calculator for .NET</p> <p><a href=\"https://ci.appveyor.com/project/Turnerj/Quickenshtein\"><img src=\"https://camo.githubusercontent.com/f8bd19156a94ff534be47484679d394b1b4bd297a9b424cdf295e05ec9d64845/68747470733a2f2f696d672e736869656c64732e696f2f6170707665796f722f63692f5475726e65726a2f517569636b656e73687465696e2f6d61737465722e737667\" alt=\"AppVeyor\"></a> <a href=\"https://codecov.io/gh/Turnerj/Quickenshtein\"><img src=\"https://camo.githubusercontent.com/f3d33f414d0ed126f1292d63e5a0b0f813cef58a7c1af07be68b5a714a1db3c9/68747470733a2f2f696d672e736869656c64732e696f2f636f6465636f762f632f6769746875622f5475726e65726a2f517569636b656e73687465696e2f6d61737465722e737667\" alt=\"Codecov\"></a> <a href=\"https://www.nuget.org/packages/Quickenshtein/\"><img src=\"https://camo.githubusercontent.com/a060bbf3ad717682e4e2ec0c8f5224fc078dc46f21bbe419c1691933d385d5df/68747470733a2f2f696d672e736869656c64732e696f2f6e756765742f762f517569636b656e73687465696e2e737667\" alt=\"NuGet\"></a></p> <h2>Performance</h2> <p>Quickenshtein gets its speed and memory efficiency from a number of different optimizations. To get the most performance out of the library, you will need .NET Core 3 or higher as this has support for hardware intrinsics.</p> <p>Quickenshtein takes advantage of the following hardware intrinsics. On any recent x86 system, you will likely have these available.</p> <ul><li><a href=\"https://en.wikipedia.org/wiki/SSE2#CPU_support\">SSE2</a></li><li><a href=\"https://en.wikipedia.org/wiki/SSE4#Supporting_CPUs\">SSE4.1</a></li><li><a href=\"https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2\">AVX2</a></li></ul> <p>If your computer doesn't have one of the hardware intrinsics available, Quickenshtein will still work - just slower than optimal.</p> <h2>Multi-Threading</h2> <p>By default, Quickenshtein performs in single-threaded mode as this mode performs best for small to medium size strings while having no memory allocations. When dealing with huge strings of 8000 characters or more, it may be useful to switch to multi-threaded mode. In this mode, the calculation is broken up and shared between multiple cores in a system.</p> <p>Multi-threading is especially useful for systems without hardware intrinsics or for .NET Framework as shown in the table below where it provided a 3x performance improvement.</p>  <p>To enable threading, you can pass in <code>CalculationOptions.DefaultWithThreading</code> to <code>Levenshtein.GetDistance()</code> or configure your own <code>CalculationOptions</code> with settings that work best for you.</p> <p><em>Note: Multi-threading is not allocation free (unlike single-threading mode) and will allocate a small amount depending on the number of threads used.</em></p> <h2>Benchmarking</h2> <p>There are a number of benchmarks in the repository that you can run on your system to see how well Quickenshtein performs.</p> <p>Most of these benchmarks...</p> <ul><li>Run .NET Framework and .NET Core so you can see how the performance changes between them</li><li>Compare against a simple baseline Levenshtein Distance implementation with no specific optimizations</li><li>Compare against <a href=\"https://github.com/DanHarltey/Fastenshtein/\">Fastenshtein</a>, one of the other fast .NET Levenshtein Distance implementations</li></ul> <p>You can view results to these benchmarks at the links below:</p> <ul><li><a href=\"https://github.com/Turnerj/Quickenshtein/blob/main/docs/OverallComparison.md\">Benchmarking against Fastenshtein</a></li><li><a href=\"https://github.com/Turnerj/Quickenshtein/blob/main/docs/StringScenarios.md\">Isolated Benchmarks</a></li><li><a href=\"https://github.com/Turnerj/Quickenshtein/blob/main/docs/IntrinsicsPerformance.md\">Intrinsics Performance</a></li></ul> <h2>Example Usage</h2> <pre>using Quickenshtein;\n\n// Common usage (uses default CalculationOptions with threading disabled)\nvar distance1 = Levenshtein.GetDistance(\"Saturday\", \"Sunday\");\n\n// Enable threading\nvar distance2 = Levenshtein.GetDistance(\"Saturday\", \"Sunday\", CalculationOptions.DefaultWithThreading);\n\n// Custom calculation options (helps with tuning for your specific workload and environment - you should benchmark your configurations on your system)\nvar distance3 = Levenshtein.GetDistance(\"Saturday\", \"Sunday\", new CalculationOptions {\n    EnableThreadingAfterXCharacters = 10000,\n    MinimumCharactersPerThread = 25000\n});</pre>",
  "mainContentOfPage": {
    "@context": "https://schema.org",
    "@type": "WebPageElement",
    "cssSelector": "#readme>div:nth-child(3)>article"
  }
}