Internet-Draft Entropy Values May 2022
Li Expires 13 November 2022 [Page]
Workgroup:
MPLS Working Group
Internet-Draft:
draft-li-mpls-entropy-01
Published:
Intended Status:
Informational
Expires:
Author:
T. Li
Juniper Networks

Entropy Values

Abstract

Equal Cost Multi-Path (ECMP) forwarding is an essential function in distributing traffic across parallel paths. Packets within a flow must be kept on a single path to avoid reordering, while different flows must be distributed across paths to achieve parallelism.

Previously, MPLS has addressed this through the use of an entropy label, providing up to 20 bits of entropy that can be added to the label stack to distinguish different flows. [RFC6790] With the interest in MPLS Network Actions, there are proposals to embedding entropy into alternate structures, so it is an appropriate time to consider how many bits should be used for entropy in the future. [I-D.bocci-mpls-miad-adi-requirements][I-D.andersson-mpls-mna-fwk]

In this document, we examine the question of how to provide adequate entropy through a simple stochastic simulation. This is not intended to be a comprehensive and extensive treatise, but rather a simple investigation to build intuition into the issues.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 13 November 2022.

Table of Contents

1. Introduction

Equal Cost Multi-Path (ECMP) forwarding is an essential function in distributing traffic across parallel paths. Packets within a flow must be kept on a single path to avoid reordering, while different flows must be distributed across paths to achieve parallelism.

Previously, MPLS has addressed this through the use of an entropy label, providing up to 20 bits of entropy that can be added to the label stack to distinguish different flows. [RFC6790] With the interest in MPLS Network Actions, there are proposals to embedding entropy into alternate structures, so it is an appropriate time to consider how many bits should be used for entropy in the future. [I-D.bocci-mpls-miad-adi-requirements][I-D.andersson-mpls-mna-fwk]

In this document, we examine the question of how to provide adequate entropy through a simple stochastic simulation. This is not intended to be a comprehensive and extensive treatise, but rather a simple investigation to build intuition into the issues.

1.1. Model

In a typical ECMP situation, a Label Edge Router (LER) [RFC4221] would examine incoming traffic and use values in the incoming packet to compute an Entropy Label (EL). This would be placed in the label stack. Subsequent routers on the path could use this EL as input to a hashing function and the resulting hash value would then be used to select one of the possible output paths. If the entropy label and the hashing function are implemented correctly, packets would be uniformly distributed across the possible paths. If packet sizes are uniformly distributed, this would also imply that bandwidth would be uniformly distributed across paths.

Moving forward, we expect that entropy will continue to be carried in the label stack, but may not occupy a Label field inside of a Label Stack Entry (LSE) [RFC3032]. We generalize this to an Entropy Value (EV) and seek to understand the necessary size of this field.

1.2. Methodology

To study this, we employ a pseudo-random number generator to create Entropy Values and then we apply a hashing function to them. Ideally, the results of the hash will be uniformly distributed. Error metrics are used to characterize how far results are from this ideal.

2. Randomness

Real hardware does not use a random number generator to create entropy. Doing so would require mapping each flow to an entropy value and this would be painful to implement due to scale. Rather, implementations that we are familiar with use a hashing function on the packet header to create an entropy value. Ideally, the results of this would be uniformly distributed. However, we cannot assume that all traffic will be quite so cooperative.

Conversely, we also cannot assume that the LER will generate no entropy. If an LER does nothing except place a constant in the entropy value, no downstream hashing function will be able to extract any entropy.

To model this, we intentionally choose an imperfect random number generator, specifically, we choose one that has a normal distribution (i.e., Gaussian or bell curve). This should produce many different entropy values and if the hashing function is doing its job, the resulting entropy should result in a uniform distribution of hash values.

3. Hashing

Similarly, there is no standardized way of implementing the hash function that downstream routers will use to convert an entropy value into a hash value. In cases where the entropy value has more bits than the hash value can support, it is up to the hashing function to fold the entropy from all of the entropy value bits into the hash value.

Thus, we consider a number of hashing functions:

Some of these hash functions work better than others. Additional hash functions could easily be added.

To simplify our simulation, we have primarily focused on hashes that have a range that is a power of two. However, in a practical application, that is not appropriate and the range must be an arbitrary integer, so that we can support n-way ECMP. This is discussed further below.

4. Future Hardware Considerations

When ECMP was first implemented, 4-way ECMP was the norm. In the early days of sparse networks, that was frequently adequate. As the number of interfaces on a router has grown, ECMP has followed suit, and 128-way and 256-way ECMP is now in production.

While hardware has provided an impetetus to grow the breadth of ECMP, it is also suggesting that there is an upper bound to what needs to be supported. It is fairly clear that with current networking trends, we will never need more paths from a single router than it has physical interfaces. While logically, more would be possible, the growth in bandwidth has driven the need for ECMP and that bandwidth is most easily provided through the use of non-fractional interfaces.

Concurrently, we note a trend away from building multi-chassis routers with an arbitrary number of physical interfaces, so this implies a bound of around 1000 physical interface per router for the foreseable future.

Hardware is also most conveniently impelemented on power of two boundaries. Thus, we investigate ECMP from 4-way to 4096-way, in powers of two.

5. Results

For hash values from 8 to 12 bits, we consider entropy values with 0 to 5 additional bits (i.e., 8 to 17 bits). We generate enough random samples so that each hash bucket should ideally have 1000 entries.

The error for each has bucket is then difference between the number of hash results and 1000. We then compute the following error metrics for each combination of algorithm, hash value bits, and additional bits:

We then summarize the results based on the number of additional bits and on algorithm, computing the mean of each of the above metrics.

6. Conclusion

The arguments in Section 4 suggest that 4096-way ECMP will be sufficient for some time to come. This suggests that a base of 12 bits would be a good start.

We arbitrarily select a MAPE of 5.0 as a threshold of acceptability. This represents a distribution that is within 5% of our goal of a uniform distribution.

A solution to the issue would be a number of bits entropy value and a set of hashing algorithms that would provide acceptable performance over the full range of hash values, from 2 to 12 bits.

We observe that for some entropy values, we get unacceptable solutions across all algorithms:

For 2-way ECMP with 0 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add | 687.0 | 687.0 | 68.7 |
|      mask | 378.0 | 378.0 | 37.8 |
|       xor | 687.0 | 687.0 | 68.7 |
| crc-ccitt | 687.0 | 687.0 | 68.7 |
|       md4 | 687.0 | 687.0 | 68.7 |
|       md5 | 376.0 | 376.0 | 37.6 |
|      sha1 | 378.0 | 378.0 | 37.8 |
|    sha256 | 378.0 | 378.0 | 37.8 |
+-----------+-------+-------+------+

Clearly, a 2 bit entropy value is not acceptable.

We also observe that some algorithms perform better than others:

For 16-way ECMP with 1 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  46.0 |  35.0 |  3.5 |
|      mask |  30.1 |  27.2 |  2.7 |
|       xor |  48.3 |  43.9 |  4.4 |
| crc-ccitt |  24.4 |  19.9 |  2.0 |
|       md4 | 234.5 | 197.4 | 19.7 |
|       md5 | 262.5 | 196.1 | 19.6 |
|      sha1 | 183.4 | 150.0 | 15.0 |
|    sha256 | 281.1 | 251.2 | 25.1 |
+-----------+-------+-------+------+

Understanding why some algorithms perform poorly is not understood at this time and a subject for future research.

Once we reach about 10 bits of entropy value, most algorithms work well, most of the time:

For 1024-way ECMP with 0 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 32.2 | 25.8 |  2.6 |
|      mask | 32.6 | 26.2 |  2.6 |
|       xor | 33.5 | 26.6 |  2.7 |
| crc-ccitt | 31.2 | 24.9 |  2.5 |
|       md4 | 55.4 | 44.4 |  4.4 |
|       md5 | 55.9 | 44.7 |  4.5 |
|      sha1 | 57.2 | 45.4 |  4.5 |
|    sha256 | 57.0 | 46.1 |  4.6 |
+-----------+------+------+------+

For 256-way ECMP with 2 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 30.3 | 23.9 |  2.4 |
|      mask | 32.6 | 26.2 |  2.6 |
|       xor | 28.9 | 23.1 |  2.3 |
| crc-ccitt | 35.2 | 28.0 |  2.8 |
|       md4 | 59.0 | 46.5 |  4.6 |
|       md5 | 54.7 | 44.0 |  4.4 |
|      sha1 | 56.8 | 44.3 |  4.4 |
|    sha256 | 54.5 | 44.1 |  4.4 |
+-----------+------+------+------+

If we look at summaries by additional bits and by algorithm we see:

Error per added bit
+------------+-------+-------+-------+
| Added bits | MRMSE |  MMAE | MMAPE |
+------------+-------+-------+-------+
|          0 | 171.0 | 150.6 |  15.1 |
|          1 | 115.8 | 100.2 |  10.0 |
|          2 |  74.9 |  61.6 |   6.2 |
|          3 |  58.5 |  49.0 |   4.9 |
|          4 |  51.1 |  43.2 |   4.3 |
|          5 |  47.1 |  39.8 |   4.0 |
+------------+-------+-------+-------+

Error per algorithm
+-----------+-------+-------+-------+
| Algorithm | MRMSE |  MMAE | MMAPE |
+-----------+-------+-------+-------+
|       add |  52.7 |  45.8 |   4.6 |
|      mask |  34.1 |  28.6 |   2.9 |
|       xor |  54.0 |  48.0 |   4.8 |
| crc-ccitt |  48.7 |  43.2 |   4.3 |
|       md4 | 137.7 | 119.8 |  12.0 |
|       md5 | 131.3 | 110.7 |  11.1 |
|      sha1 | 110.5 |  93.0 |   9.3 |
|    sha256 | 122.2 | 103.3 |  10.3 |
+-----------+-------+-------+-------+

This leads us to propose that use at least 4 additional bits for our entropy value, resulting in total of 16 (12 + 4). Please note that this is a minimum. Most proposals are proposing higher numbers of bits, thus exceeding this minimum.

7. Validation

To further validate that 16 bits of entropy is sufficient, we also simulated hashing with a non-power of two result to support n-way ECMP. To map the hash to a range of n possible results, we performed a 'mod n' operation after each of the above hashes. We then selected 10 random n values that are not powers of two and repeated the above experiment.

In this step, we generated 16 bits of entropy, hashed that and then folded it to down to 16 bits. We then applied the 'mod n' operation to get down to a range of n results. In these circumstances, the 'add', 'mask', and 'xor' hashes are all identical and result in no changes, as they simply are the identity function on 16 bits. The 'add' and 'xor' functions were removed for efficiency.

The results demonstrate that the 'mask' (identity) function performs adequately in this circumstance, again supporting the claim that 16 bits is sufficient entropy.

For 3023-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  31.8 |  25.6 |  2.6 |
| crc-ccitt | 238.2 | 191.1 | 19.1 |
|       md4 | 320.7 | 254.9 | 25.5 |
|       md5 | 319.7 | 253.8 | 25.4 |
|      sha1 | 329.1 | 262.8 | 26.3 |
|    sha256 | 328.0 | 261.6 | 26.2 |
+-----------+-------+-------+------+

8. Security Considerations

No security issues are discussed in this document.

9. IANA Considerations

This document makes no requests of IANA.

10. References

10.1. Normative References

[RFC6790]
Kompella, K., Drake, J., Amante, S., Henderickx, W., and L. Yong, "The Use of Entropy Labels in MPLS Forwarding", RFC 6790, DOI 10.17487/RFC6790, , <https://www.rfc-editor.org/info/rfc6790>.
[RFC4221]
Nadeau, T., Srinivasan, C., and A. Farrel, "Multiprotocol Label Switching (MPLS) Management Overview", RFC 4221, DOI 10.17487/RFC4221, , <https://www.rfc-editor.org/info/rfc4221>.
[RFC3032]
Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack Encoding", RFC 3032, DOI 10.17487/RFC3032, , <https://www.rfc-editor.org/info/rfc3032>.

10.2. Informative References

[I-D.bocci-mpls-miad-adi-requirements]
Bocci, M. and S. Bryant, "Requirements for MPLS Network Action Indicators and MPLS Ancillary Data", Work in Progress, Internet-Draft, draft-bocci-mpls-miad-adi-requirements-04, , <https://www.ietf.org/archive/id/draft-bocci-mpls-miad-adi-requirements-04.txt>.
[I-D.andersson-mpls-mna-fwk]
Andersson, L., Bryant, S., Bocci, M., and T. Li, "MPLS Network Actions Framework", Work in Progress, Internet-Draft, draft-andersson-mpls-mna-fwk-01, , <https://www.ietf.org/archive/id/draft-andersson-mpls-mna-fwk-01.txt>.

Appendix A. Source Code

#!/usr/bin/python3
#
# Copyright 2022, Juniper Networks, Inc.
# Tony Li (tli)
#
# Statistical study on the number of entropy bits necessary.
#
# Equal Cost Multi-Path (ECMP) forwarding directs traffic down
# multiple paths. An entropy label is included in the MPLS label
# stack or in the MPLS Network Action sub-stack to help
# implementations keep individual flows on a consistent path but
# spread different flows across different paths.
#
# The entropy label is supposed to be used by an implementation
# as input to a hash function, producing a hash value that is an
# index into the set of paths. If the hash values are uniformly
# distributed, then the paths will be equally used. If the hash
# function is poor or there is not enough entropy in the entropy
# label, then the hash values will not be uniformly distribute.
#
# To gain some intuition about how many bits of entropy are
# necessary, we simulate the hashing of a large number of random
# values and then measure the uniformity of the results.
#

import crcmod
from numpy import random, round
import matplotlib.pyplot as plt
import math
from tabulate import tabulate
import binascii
import hashlib

# To get an interesting volume of data, we need about this many
# samples per bucket.
DEPTH = 1000

# ECMP is typically up to N way, where N is 2 ** P.  We study a
# range of P values to get an understanding of the number of
# necessary bits as a function of P.

P_RANGE = range(1, 13)
#P_RANGE = range(2, 6)

# Consider this many additional bits of entropy, above and beyond
# P.
MORE_BITS_RANGE = range(0, 6)

# Format for floating point numbers
FLT = '%.1f'

class HashFunc(object):
   def __init__ (self, p, name):
      self.bits = p
      self.name = name
      # Mask of bits inside of the target range
      self.lowmask = (2 ** p - 1)
      # Mask of bits outside of the target range
      self.highmask = ~self.lowmask

   def hashfunc(self, v):
      raise NotImplementedError()

class AddHash(HashFunc):
   def __init__(self, p):
      super().__init__(p, 'add')

   def hashfunc(self, v):
      old = v
      if self.highmask & v:
         v = (v & self.lowmask) + ((v & self.highmask) >>
                                   self.bits)
      return v & self.lowmask

class MaskHash(HashFunc):
   def __init__(self, p):
      super().__init__(p, 'mask')

   def hashfunc(self, v):
      return v & self.lowmask

class XorHash(HashFunc):
   def __init__(self, p):
      super().__init__(p, 'xor')

   def hashfunc(self, v):
      while self.highmask & v:
         v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                   self.bits)
      return v

class CrcHash(HashFunc):
   # CRC-CCITT
   def __init__(self, p):
      super().__init__(p, 'crc-ccitt')

   def hashfunc(self, v):
      v = binascii.crc_hqx(v.to_bytes(4, 'little'), 0)
      while self.highmask & v:
         v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                   self.bits)
      return v

class Md4Hash(HashFunc):
   def __init__(self, p):
      super().__init__(p, 'md4')
      self.func = hashlib.new('md4')

   def hashfunc(self, v):
      m = self.func.copy()
      m.update(v.to_bytes(4, 'little'))
      d = m.digest()
      v = int.from_bytes(d, 'little')
      while self.highmask & v:
         v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                   self.bits)
      return v

class Md5Hash(HashFunc):
   def __init__(self, p):
      super().__init__(p, 'md5')
      self.func = hashlib.md5()

   def hashfunc(self, v):
      m = self.func.copy()
      m.update(v.to_bytes(4, 'little'))
      d = m.digest()
      v = int.from_bytes(d, 'little')
      while self.highmask & v:
         v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                   self.bits)
      return v

class Sha1Hash(HashFunc):
   def __init__(self, p):
      super().__init__(p, 'sha1')
      self.func = hashlib.sha1()

   def hashfunc(self, v):
      m = self.func.copy()
      m.update(v.to_bytes(4, 'little'))
      d = m.digest()
      v = int.from_bytes(d, 'little')
      while self.highmask & v:
         v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                   self.bits)
      return v

class Sha256Hash(HashFunc):
   def __init__(self, p):
      super().__init__(p, 'sha256')
      self.func = hashlib.sha256()

   def hashfunc(self, v):
      m = self.func.copy()
      m.update(v.to_bytes(4, 'little'))
      d = m.digest()
      v = int.from_bytes(d, 'little')
      while self.highmask & v:
         v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                   self.bits)
      return v

def rmse( error ):
   # Compute the root mean square error of this run.
   errsq = [ err ** 2 for err in error ]
   rmse = math.sqrt( sum( errsq ) / len( error ) )
   return rmse

def mae( error ):
   # Compute the mean average error of this run.
   mae = sum( [ abs( err ) for err in error ] ) / len( error )
   return mae

def mape( error ):
   # Compute the mean average percentage error of this run.
   mape = 0.0
   for ( ind, err ) in enumerate( error ):
      if err != 0.0:
         mape += abs( err / DEPTH)
   mape = mape / len( error )
   return mape * 100.0

def mean(vec):
   return sum(vec)/len(vec)

def samples(buckets, bits):
   # Uniform distribution
   #samples = random.randint(0, (2 ** bits), buckets * DEPTH)
   # Normal distribution
   # This emulates a 'poor' random number generator
   floats = random.normal(2 ** (bits - 1),
                          scale=2 ** (bits - 3),
                          size=(buckets * DEPTH))
   rounded = round(floats)
   samples = [ int(x) for x in rounded ]
   samples = [ x if x > 0 else -x for x in samples ]
   samples = [ x if x < 2 ** bits else (2 ** bits - 1) for x in
               samples ]
   # Debugging
   # print (samples)
   #
   #count, bins, ignored = plt.hist(samples, bins = 2 ** bits)
   #plt.title('Random samples using %d bits of entropy' %
   #          bits)
   #plt.show()
   #
   return samples

headers = ['Hash', 'RMSE', 'MAE', 'MAPE']
summary_headers = ['Added bits', 'MRMSE', 'MMAE', 'MMAPE']
algo_summary_headers = ['Algorithm', 'MRMSE', 'MMAE', 'MMAPE']

def power_of_two():

   def simulate(p, bits):
      total = p + bits
      entropy = samples(2 ** p, total)
      hashes = [ AddHash(p), MaskHash(p), XorHash(p), CrcHash(p),
                 Md4Hash(p), Md5Hash(p), Sha1Hash(p),
                 Sha256Hash(p) ]
      results = []
      for h in hashes:
         hashvals = [ h.hashfunc(x) for x in entropy ]
         # print(hashvals)
         buckets = [ 0 ] * 2 ** p
         for x in hashvals:
            buckets[x] += 1
         # print(h.name, buckets, sum(buckets))
         error = [ DEPTH - x for x in buckets ]
         # print(error)
         # print(rmse(error), mae(error), mape(error))
         results.append( [ h.name, FLT % rmse(error), FLT %
                           mae(error), FLT % mape(error) ] )
      return results

   summary = []
   per_algo = {}
   for more in MORE_BITS_RANGE:
      per_bit = []
      for p in P_RANGE:
         print( 'For %d-way ECMP with %d more bits of entropy:' %
                (2 ** p, more))
         results = simulate(p, p + more)
         print(tabulate(results, headers=headers, tablefmt='pretty',
                        stralign='right'))
         rmsevec = [ float(x[1]) for x in results ]
         maevec = [ float(x[2]) for x in results ]
         mapevec = [ float(x[3]) for x in results ]
         per_bit.append([FLT % mean(rmsevec), FLT % mean(maevec),
                         FLT % mean(mapevec)])
         print()

         for result in results:
            if not per_algo.get(result[0]):
               per_algo[result[0]] = [ [], [], [] ]
            for i in range(1,4):
               per_algo[result[0]][i-1].append(float(result[i]))

      rmsevec = [ float(x[0]) for x in per_bit ]
      maevec = [ float(x[1]) for x in per_bit ]
      mapevec = [ float(x[2]) for x in per_bit ]
      summary.append([more, FLT % mean(rmsevec), FLT % mean(maevec),
                      FLT % mean(mapevec)])

   print('Error per added bit')
   print(tabulate(summary, headers=summary_headers,
                  tablefmt='pretty', stralign='right'))

   algo_summary = []
   for algo in per_algo:
      algo_summary.append([algo,
                           FLT % mean(per_algo[algo][0]),
                           FLT % mean(per_algo[algo][1]),
                           FLT % mean(per_algo[algo][2])])
   print('\nError per algorithm')
   print(tabulate(algo_summary, headers=algo_summary_headers,
                  tablefmt='pretty', stralign='right'))

def multibit():
   # Analyze hashes for non-power-of-two ranges.
   ENTROPY_BITS = 16
   SAMPLES = 10

   # For this case, Add, Xor, and Mask are equivalent: a NOP
   hashes = [ #AddHash(ENTROPY_BITS), XorHash(ENTROPY_BITS),
      MaskHash(ENTROPY_BITS), CrcHash(ENTROPY_BITS),
      Md4Hash(ENTROPY_BITS), Md5Hash(ENTROPY_BITS),
      Sha1Hash(ENTROPY_BITS), Sha256Hash(ENTROPY_BITS) ]

   def waygen():
      start = 2 ** min(P_RANGE)
      stop = 2 ** max(P_RANGE)
      for x in range(start, stop):
         # Don't bother with powers of two again.
         if x & (x - 1) != 0:
            yield x

   def simulate(nways):
      entropy = samples(nways, ENTROPY_BITS)
      results = []
      for h in hashes:
         hashvals = [ h.hashfunc(x) for x in entropy ]
         # print(hashvals)
         buckets = [ 0 ] * nways
         for x in hashvals:
            buckets[x % nways] += 1
         #print(h.name, buckets, sum(buckets))
         error = [ DEPTH - x for x in buckets ]
         # print(error)
         results.append( [ h.name, FLT % rmse(error), FLT %
                           mae(error), FLT % mape(error) ] )
      return results

   per_way = []
   nways = [ x for x in waygen() ]
   for nway in random.choice(nways, size=SAMPLES, replace=False):
      results = simulate(nway)
      print('\nFor %d-way ECMP, %d bits of entropy:' %
            (nway, ENTROPY_BITS))
      print(tabulate(results, headers=headers, tablefmt='pretty',
                     stralign='right'))
      # print('%d-way' % nway)

      rmsevec = [ float(x[1]) for x in results ]
      maevec = [ float(x[2]) for x in results ]
      mapevec = [ float(x[3]) for x in results ]
      per_way.append([nway, FLT % mean(rmsevec),
                      FLT % mean(maevec), FLT % mean(mapevec)])

   summary_headers = ['n-ways', 'MRMSE', 'MMAE', 'MMAPE']
   print('\nFor n-way ECMP, with %d bits of entropy, '
         'across hash functions' % ENTROPY_BITS)
   print(tabulate(per_way, headers=summary_headers,
                  tablefmt='pretty',
                  stralign='right'))

power_of_two()
multibit()

Appendix B. Full results

For 2-way ECMP with 0 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add | 687.0 | 687.0 | 68.7 |
|      mask | 378.0 | 378.0 | 37.8 |
|       xor | 687.0 | 687.0 | 68.7 |
| crc-ccitt | 687.0 | 687.0 | 68.7 |
|       md4 | 687.0 | 687.0 | 68.7 |
|       md5 | 376.0 | 376.0 | 37.6 |
|      sha1 | 378.0 | 378.0 | 37.8 |
|    sha256 | 378.0 | 378.0 | 37.8 |
+-----------+-------+-------+------+

For 4-way ECMP with 0 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add | 376.0 | 315.5 | 31.6 |
|      mask |  39.0 |  36.5 |  3.7 |
|       xor | 470.9 | 444.0 | 44.4 |
| crc-ccitt |  56.5 |  48.5 |  4.9 |
|       md4 | 990.4 | 936.0 | 93.6 |
|       md5 | 532.6 | 504.0 | 50.4 |
|      sha1 | 593.1 | 500.0 | 50.0 |
|    sha256 | 358.4 | 260.5 | 26.1 |
+-----------+-------+-------+------+

For 8-way ECMP with 0 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add | 167.9 | 150.8 | 15.1 |
|      mask |  35.5 |  28.8 |  2.9 |
|       xor | 227.2 | 213.2 | 21.3 |
| crc-ccitt | 443.4 | 442.5 | 44.2 |
|       md4 | 355.3 | 277.8 | 27.8 |
|       md5 | 292.2 | 252.2 | 25.2 |
|      sha1 | 321.3 | 284.8 | 28.5 |
|    sha256 | 439.9 | 377.8 | 37.8 |
+-----------+-------+-------+------+

For 16-way ECMP with 0 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  75.4 |  60.1 |  6.0 |
|      mask |  40.2 |  35.8 |  3.6 |
|       xor | 120.2 | 112.0 | 11.2 |
| crc-ccitt |  33.6 |  28.6 |  2.9 |
|       md4 | 423.0 | 316.2 | 31.6 |
|       md5 | 374.6 | 272.2 | 27.2 |
|      sha1 | 345.7 | 282.9 | 28.3 |
|    sha256 | 407.9 | 285.5 | 28.5 |
+-----------+-------+-------+------+

For 32-way ECMP with 0 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  50.9 |  44.6 |  4.5 |
|      mask |  32.1 |  26.8 |  2.7 |
|       xor |  58.5 |  48.7 |  4.9 |
| crc-ccitt |  32.7 |  29.1 |  2.9 |
|       md4 | 251.6 | 191.8 | 19.2 |
|       md5 | 245.9 | 202.4 | 20.2 |
|      sha1 | 274.4 | 228.1 | 22.8 |
|    sha256 | 257.5 | 211.2 | 21.1 |
+-----------+-------+-------+------+

For 64-way ECMP with 0 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  33.5 |  26.2 |  2.6 |
|      mask |  25.1 |  18.7 |  1.9 |
|       xor |  33.1 |  27.1 |  2.7 |
| crc-ccitt |  32.9 |  26.4 |  2.6 |
|       md4 | 204.7 | 168.9 | 16.9 |
|       md5 | 173.4 | 140.5 | 14.0 |
|      sha1 | 227.9 | 182.1 | 18.2 |
|    sha256 | 165.2 | 131.5 | 13.2 |
+-----------+-------+-------+------+

For 128-way ECMP with 0 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  27.8 |  22.4 |  2.2 |
|      mask |  30.9 |  24.8 |  2.5 |
|       xor |  34.2 |  26.9 |  2.7 |
| crc-ccitt |  33.2 |  27.0 |  2.7 |
|       md4 | 131.9 | 105.5 | 10.5 |
|       md5 | 123.1 | 100.9 | 10.1 |
|      sha1 | 129.5 | 106.8 | 10.7 |
|    sha256 | 133.4 | 105.0 | 10.5 |
+-----------+-------+-------+------+

For 256-way ECMP with 0 more bits of entropy:
+-----------+-------+------+------+
|      Hash |  RMSE |  MAE | MAPE |
+-----------+-------+------+------+
|       add |  29.6 | 23.8 |  2.4 |
|      mask |  31.4 | 25.1 |  2.5 |
|       xor |  32.5 | 26.1 |  2.6 |
| crc-ccitt |  31.2 | 24.9 |  2.5 |
|       md4 |  94.6 | 76.1 |  7.6 |
|       md5 | 100.6 | 81.3 |  8.1 |
|      sha1 | 102.3 | 81.1 |  8.1 |
|    sha256 |  98.6 | 79.6 |  8.0 |
+-----------+-------+------+------+

For 512-way ECMP with 0 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.4 | 24.9 |  2.5 |
|      mask | 32.2 | 25.8 |  2.6 |
|       xor | 34.1 | 27.0 |  2.7 |
| crc-ccitt | 30.7 | 25.0 |  2.5 |
|       md4 | 72.6 | 58.2 |  5.8 |
|       md5 | 71.7 | 56.6 |  5.7 |
|      sha1 | 77.6 | 62.1 |  6.2 |
|    sha256 | 72.2 | 57.6 |  5.8 |
+-----------+------+------+------+

For 1024-way ECMP with 0 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 32.2 | 25.8 |  2.6 |
|      mask | 32.6 | 26.2 |  2.6 |
|       xor | 33.5 | 26.6 |  2.7 |
| crc-ccitt | 31.2 | 24.9 |  2.5 |
|       md4 | 55.4 | 44.4 |  4.4 |
|       md5 | 55.9 | 44.7 |  4.5 |
|      sha1 | 57.2 | 45.4 |  4.5 |
|    sha256 | 57.0 | 46.1 |  4.6 |
+-----------+------+------+------+

For 2048-way ECMP with 0 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.5 | 25.1 |  2.5 |
|      mask | 31.3 | 25.0 |  2.5 |
|       xor | 31.9 | 25.6 |  2.6 |
| crc-ccitt | 31.9 | 25.2 |  2.5 |
|       md4 | 46.1 | 36.8 |  3.7 |
|       md5 | 45.8 | 36.4 |  3.6 |
|      sha1 | 46.9 | 38.0 |  3.8 |
|    sha256 | 47.0 | 37.6 |  3.8 |
+-----------+------+------+------+

For 4096-way ECMP with 0 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.4 | 24.9 |  2.5 |
|      mask | 31.5 | 24.9 |  2.5 |
|       xor | 31.2 | 24.9 |  2.5 |
| crc-ccitt | 31.7 | 25.2 |  2.5 |
|       md4 | 39.0 | 31.2 |  3.1 |
|       md5 | 39.8 | 31.9 |  3.2 |
|      sha1 | 39.1 | 31.1 |  3.1 |
|    sha256 | 39.2 | 31.2 |  3.1 |
+-----------+------+------+------+

For 2-way ECMP with 1 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add | 269.0 | 269.0 | 26.9 |
|      mask |   3.0 |   3.0 |  0.3 |
|       xor |  94.0 |  94.0 |  9.4 |
| crc-ccitt |  94.0 |  94.0 |  9.4 |
|       md4 | 761.0 | 761.0 | 76.1 |
|       md5 | 381.0 | 381.0 | 38.1 |
|      sha1 | 104.0 | 104.0 | 10.4 |
|    sha256 | 771.0 | 771.0 | 77.1 |
+-----------+-------+-------+------+

For 4-way ECMP with 1 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add | 153.0 | 131.5 | 13.2 |
|      mask |  34.5 |  27.0 |  2.7 |
|       xor | 161.6 | 161.0 | 16.1 |
| crc-ccitt |  36.9 |  31.0 |  3.1 |
|       md4 | 332.6 | 307.0 | 30.7 |
|       md5 | 562.9 | 443.5 | 44.4 |
|      sha1 | 219.2 | 184.0 | 18.4 |
|    sha256 | 358.9 | 325.0 | 32.5 |
+-----------+-------+-------+------+

For 8-way ECMP with 1 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  74.6 |  67.2 |  6.7 |
|      mask |  17.3 |  15.8 |  1.6 |
|       xor |  76.0 |  75.5 |  7.5 |
| crc-ccitt | 167.8 | 166.5 | 16.6 |
|       md4 | 458.2 | 408.5 | 40.9 |
|       md5 | 340.7 | 247.8 | 24.8 |
|      sha1 | 487.6 | 382.2 | 38.2 |
|    sha256 | 355.5 | 282.5 | 28.2 |
+-----------+-------+-------+------+

For 16-way ECMP with 1 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  46.0 |  35.0 |  3.5 |
|      mask |  30.1 |  27.2 |  2.7 |
|       xor |  48.3 |  43.9 |  4.4 |
| crc-ccitt |  24.4 |  19.9 |  2.0 |
|       md4 | 234.5 | 197.4 | 19.7 |
|       md5 | 262.5 | 196.1 | 19.6 |
|      sha1 | 183.4 | 150.0 | 15.0 |
|    sha256 | 281.1 | 251.2 | 25.1 |
+-----------+-------+-------+------+

For 32-way ECMP with 1 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  35.9 |  28.9 |  2.9 |
|      mask |  31.2 |  26.1 |  2.6 |
|       xor |  32.6 |  26.8 |  2.7 |
| crc-ccitt |  40.0 |  32.5 |  3.3 |
|       md4 | 190.5 | 164.8 | 16.5 |
|       md5 | 194.2 | 152.2 | 15.2 |
|      sha1 | 188.4 | 155.7 | 15.6 |
|    sha256 | 145.6 | 121.2 | 12.1 |
+-----------+-------+-------+------+

For 64-way ECMP with 1 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  34.4 |  27.7 |  2.8 |
|      mask |  31.3 |  26.4 |  2.6 |
|       xor |  38.2 |  31.6 |  3.2 |
| crc-ccitt |  32.3 |  24.3 |  2.4 |
|       md4 | 127.3 |  96.7 |  9.7 |
|       md5 | 123.8 | 101.7 | 10.2 |
|      sha1 | 150.6 | 122.3 | 12.2 |
|    sha256 | 145.4 | 108.1 | 10.8 |
+-----------+-------+-------+------+

For 128-way ECMP with 1 more bits of entropy:
+-----------+-------+------+------+
|      Hash |  RMSE |  MAE | MAPE |
+-----------+-------+------+------+
|       add |  32.2 | 25.5 |  2.5 |
|      mask |  33.4 | 25.7 |  2.6 |
|       xor |  34.5 | 27.6 |  2.8 |
| crc-ccitt |  27.7 | 20.7 |  2.1 |
|       md4 |  94.6 | 77.5 |  7.7 |
|       md5 | 105.4 | 83.5 |  8.4 |
|      sha1 |  96.2 | 77.4 |  7.7 |
|    sha256 | 100.2 | 75.9 |  7.6 |
+-----------+-------+------+------+

For 256-way ECMP with 1 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.0 | 24.9 |  2.5 |
|      mask | 30.8 | 25.1 |  2.5 |
|       xor | 28.7 | 22.5 |  2.3 |
| crc-ccitt | 33.8 | 27.6 |  2.8 |
|       md4 | 72.6 | 59.9 |  6.0 |
|       md5 | 69.4 | 55.5 |  5.5 |
|      sha1 | 72.8 | 58.6 |  5.9 |
|    sha256 | 72.7 | 56.7 |  5.7 |
+-----------+------+------+------+

For 512-way ECMP with 1 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.7 | 25.4 |  2.5 |
|      mask | 29.7 | 23.5 |  2.3 |
|       xor | 31.9 | 25.4 |  2.5 |
| crc-ccitt | 30.9 | 24.7 |  2.5 |
|       md4 | 53.8 | 43.2 |  4.3 |
|       md5 | 57.3 | 46.7 |  4.7 |
|      sha1 | 59.7 | 47.2 |  4.7 |
|    sha256 | 56.2 | 45.4 |  4.5 |
+-----------+------+------+------+

For 1024-way ECMP with 1 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.6 | 25.1 |  2.5 |
|      mask | 31.0 | 25.0 |  2.5 |
|       xor | 31.5 | 25.1 |  2.5 |
| crc-ccitt | 30.2 | 24.1 |  2.4 |
|       md4 | 45.0 | 35.4 |  3.5 |
|       md5 | 47.6 | 38.0 |  3.8 |
|      sha1 | 45.7 | 36.7 |  3.7 |
|    sha256 | 46.5 | 36.8 |  3.7 |
+-----------+------+------+------+

For 2048-way ECMP with 1 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 32.7 | 26.0 |  2.6 |
|      mask | 31.9 | 25.5 |  2.6 |
|       xor | 31.8 | 25.4 |  2.5 |
| crc-ccitt | 32.4 | 25.8 |  2.6 |
|       md4 | 39.5 | 31.8 |  3.2 |
|       md5 | 38.6 | 30.8 |  3.1 |
|      sha1 | 39.3 | 31.1 |  3.1 |
|    sha256 | 39.7 | 31.9 |  3.2 |
+-----------+------+------+------+

For 4096-way ECMP with 1 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.7 | 25.3 |  2.5 |
|      mask | 31.5 | 25.1 |  2.5 |
|       xor | 31.8 | 25.5 |  2.6 |
| crc-ccitt | 32.3 | 25.9 |  2.6 |
|       md4 | 35.9 | 28.5 |  2.9 |
|       md5 | 35.7 | 28.5 |  2.9 |
|      sha1 | 35.2 | 28.1 |  2.8 |
|    sha256 | 36.3 | 28.9 |  2.9 |
+-----------+------+------+------+

For 2-way ECMP with 2 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |   2.0 |   2.0 |  0.2 |
|      mask |  15.0 |  15.0 |  1.5 |
|       xor |  44.0 |  44.0 |  4.4 |
| crc-ccitt |  44.0 |  44.0 |  4.4 |
|       md4 | 197.0 | 197.0 | 19.7 |
|       md5 | 123.0 | 123.0 | 12.3 |
|      sha1 | 238.0 | 238.0 | 23.8 |
|    sha256 |  20.0 |  20.0 |  2.0 |
+-----------+-------+-------+------+

For 4-way ECMP with 2 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  14.1 |  12.5 |  1.2 |
|      mask |   8.2 |   8.0 |  0.8 |
|       xor |   9.5 |   7.5 |  0.8 |
| crc-ccitt |  66.0 |  57.5 |  5.8 |
|       md4 | 119.6 |  94.5 |  9.4 |
|       md5 | 553.7 | 472.0 | 47.2 |
|      sha1 | 163.8 | 134.5 | 13.5 |
|    sha256 | 366.7 | 298.5 | 29.8 |
+-----------+-------+-------+------+

For 8-way ECMP with 2 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  30.2 |  22.5 |  2.2 |
|      mask |  24.0 |  19.8 |  2.0 |
|       xor |  43.9 |  36.5 |  3.7 |
| crc-ccitt |  25.8 |  22.2 |  2.2 |
|       md4 | 245.6 | 163.5 | 16.3 |
|       md5 | 233.6 | 187.2 | 18.7 |
|      sha1 | 337.1 | 295.8 | 29.6 |
|    sha256 | 247.0 | 194.2 | 19.4 |
+-----------+-------+-------+------+

For 16-way ECMP with 2 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  23.0 |  19.0 |  1.9 |
|      mask |  33.5 |  27.9 |  2.8 |
|       xor |  26.4 |  21.5 |  2.2 |
| crc-ccitt |  26.1 |  22.4 |  2.2 |
|       md4 | 212.4 | 151.2 | 15.1 |
|       md5 | 227.4 | 174.5 | 17.5 |
|      sha1 | 180.4 | 146.9 | 14.7 |
|    sha256 | 193.3 | 155.8 | 15.6 |
+-----------+-------+-------+------+

For 32-way ECMP with 2 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  35.4 |  28.9 |  2.9 |
|      mask |  26.6 |  22.8 |  2.3 |
|       xor |  30.8 |  23.7 |  2.4 |
| crc-ccitt |  30.4 |  23.6 |  2.4 |
|       md4 | 131.6 | 107.1 | 10.7 |
|       md5 | 136.4 | 104.9 | 10.5 |
|      sha1 | 121.8 | 101.3 | 10.1 |
|    sha256 | 137.0 | 112.8 | 11.3 |
+-----------+-------+-------+------+

For 64-way ECMP with 2 more bits of entropy:
+-----------+-------+------+------+
|      Hash |  RMSE |  MAE | MAPE |
+-----------+-------+------+------+
|       add |  30.6 | 24.7 |  2.5 |
|      mask |  28.6 | 21.5 |  2.1 |
|       xor |  35.1 | 28.8 |  2.9 |
| crc-ccitt |  37.4 | 30.3 |  3.0 |
|       md4 |  95.3 | 80.0 |  8.0 |
|       md5 | 110.5 | 89.8 |  9.0 |
|      sha1 | 106.2 | 85.4 |  8.5 |
|    sha256 | 104.7 | 84.9 |  8.5 |
+-----------+-------+------+------+

For 128-way ECMP with 2 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 30.5 | 24.5 |  2.4 |
|      mask | 33.6 | 27.0 |  2.7 |
|       xor | 31.1 | 25.5 |  2.6 |
| crc-ccitt | 31.7 | 25.4 |  2.5 |
|       md4 | 69.0 | 55.2 |  5.5 |
|       md5 | 72.6 | 55.3 |  5.5 |
|      sha1 | 76.9 | 60.5 |  6.1 |
|    sha256 | 75.9 | 59.8 |  6.0 |
+-----------+------+------+------+

For 256-way ECMP with 2 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 30.3 | 23.9 |  2.4 |
|      mask | 32.6 | 26.2 |  2.6 |
|       xor | 28.9 | 23.1 |  2.3 |
| crc-ccitt | 35.2 | 28.0 |  2.8 |
|       md4 | 59.0 | 46.5 |  4.6 |
|       md5 | 54.7 | 44.0 |  4.4 |
|      sha1 | 56.8 | 44.3 |  4.4 |
|    sha256 | 54.5 | 44.1 |  4.4 |
+-----------+------+------+------+

For 512-way ECMP with 2 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 32.2 | 25.4 |  2.5 |
|      mask | 32.4 | 25.8 |  2.6 |
|       xor | 31.8 | 25.1 |  2.5 |
| crc-ccitt | 33.2 | 26.6 |  2.7 |
|       md4 | 47.0 | 36.5 |  3.7 |
|       md5 | 45.4 | 36.5 |  3.6 |
|      sha1 | 44.1 | 35.6 |  3.6 |
|    sha256 | 45.7 | 36.5 |  3.6 |
+-----------+------+------+------+

For 1024-way ECMP with 2 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.2 | 25.2 |  2.5 |
|      mask | 30.8 | 24.5 |  2.5 |
|       xor | 31.6 | 24.8 |  2.5 |
| crc-ccitt | 31.9 | 25.5 |  2.5 |
|       md4 | 39.5 | 31.4 |  3.1 |
|       md5 | 39.4 | 31.8 |  3.2 |
|      sha1 | 39.3 | 31.4 |  3.1 |
|    sha256 | 38.6 | 31.0 |  3.1 |
+-----------+------+------+------+

For 2048-way ECMP with 2 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.5 | 25.1 |  2.5 |
|      mask | 32.4 | 25.6 |  2.6 |
|       xor | 32.7 | 26.0 |  2.6 |
| crc-ccitt | 32.1 | 25.5 |  2.6 |
|       md4 | 36.4 | 29.1 |  2.9 |
|       md5 | 35.8 | 28.6 |  2.9 |
|      sha1 | 34.9 | 27.6 |  2.8 |
|    sha256 | 35.3 | 27.9 |  2.8 |
+-----------+------+------+------+

For 4096-way ECMP with 2 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.9 | 25.4 |  2.5 |
|      mask | 31.2 | 24.9 |  2.5 |
|       xor | 32.1 | 25.8 |  2.6 |
| crc-ccitt | 31.4 | 25.0 |  2.5 |
|       md4 | 33.3 | 26.7 |  2.7 |
|       md5 | 33.8 | 27.0 |  2.7 |
|      sha1 | 34.0 | 27.1 |  2.7 |
|    sha256 | 34.3 | 27.2 |  2.7 |
+-----------+------+------+------+

For 2-way ECMP with 3 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  10.0 |  10.0 |  1.0 |
|      mask |   0.0 |   0.0 |  0.0 |
|       xor |   1.0 |   1.0 |  0.1 |
| crc-ccitt |   1.0 |   1.0 |  0.1 |
|       md4 |  28.0 |  28.0 |  2.8 |
|       md5 | 372.0 | 372.0 | 37.2 |
|      sha1 |  36.0 |  36.0 |  3.6 |
|    sha256 |  99.0 |  99.0 |  9.9 |
+-----------+-------+-------+------+

For 4-way ECMP with 3 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  16.4 |  13.5 |  1.3 |
|      mask |  26.5 |  21.5 |  2.1 |
|       xor |  25.8 |  22.5 |  2.2 |
| crc-ccitt |  24.4 |  21.0 |  2.1 |
|       md4 | 411.4 | 408.5 | 40.8 |
|       md5 | 191.2 | 163.0 | 16.3 |
|      sha1 |  88.6 |  72.0 |  7.2 |
|    sha256 | 177.6 | 152.5 | 15.2 |
+-----------+-------+-------+------+

For 8-way ECMP with 3 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  29.4 |  27.0 |  2.7 |
|      mask |  34.4 |  30.0 |  3.0 |
|       xor |  38.2 |  25.5 |  2.5 |
| crc-ccitt |  35.2 |  32.0 |  3.2 |
|       md4 | 133.3 | 115.2 | 11.5 |
|       md5 | 257.9 | 179.2 | 17.9 |
|      sha1 | 139.7 | 109.5 | 10.9 |
|    sha256 | 120.1 | 108.8 | 10.9 |
+-----------+-------+-------+------+

For 16-way ECMP with 3 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  29.5 |  22.5 |  2.3 |
|      mask |  21.8 |  17.5 |  1.8 |
|       xor |  29.8 |  26.1 |  2.6 |
| crc-ccitt |  23.4 |  18.9 |  1.9 |
|       md4 | 116.8 |  83.8 |  8.4 |
|       md5 | 156.1 | 130.5 | 13.1 |
|      sha1 | 144.3 | 116.9 | 11.7 |
|    sha256 | 138.1 | 113.6 | 11.4 |
+-----------+-------+-------+------+

For 32-way ECMP with 3 more bits of entropy:
+-----------+-------+------+------+
|      Hash |  RMSE |  MAE | MAPE |
+-----------+-------+------+------+
|       add |  27.9 | 22.9 |  2.3 |
|      mask |  33.4 | 23.9 |  2.4 |
|       xor |  27.4 | 22.1 |  2.2 |
| crc-ccitt |  27.6 | 21.4 |  2.1 |
|       md4 |  90.3 | 70.6 |  7.1 |
|       md5 |  75.9 | 60.9 |  6.1 |
|      sha1 |  96.7 | 76.5 |  7.7 |
|    sha256 | 116.8 | 96.2 |  9.6 |
+-----------+-------+------+------+

For 64-way ECMP with 3 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 32.0 | 25.9 |  2.6 |
|      mask | 27.7 | 22.1 |  2.2 |
|       xor | 32.2 | 26.0 |  2.6 |
| crc-ccitt | 32.1 | 25.1 |  2.5 |
|       md4 | 64.5 | 51.1 |  5.1 |
|       md5 | 76.9 | 60.4 |  6.0 |
|      sha1 | 77.1 | 62.1 |  6.2 |
|    sha256 | 78.3 | 62.7 |  6.3 |
+-----------+------+------+------+

For 128-way ECMP with 3 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 29.4 | 24.2 |  2.4 |
|      mask | 32.1 | 25.5 |  2.5 |
|       xor | 30.7 | 24.7 |  2.5 |
| crc-ccitt | 33.8 | 26.7 |  2.7 |
|       md4 | 52.7 | 42.8 |  4.3 |
|       md5 | 55.5 | 45.0 |  4.5 |
|      sha1 | 60.3 | 48.5 |  4.8 |
|    sha256 | 57.0 | 46.2 |  4.6 |
+-----------+------+------+------+

For 256-way ECMP with 3 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.2 | 25.5 |  2.5 |
|      mask | 30.3 | 24.3 |  2.4 |
|       xor | 32.6 | 26.4 |  2.6 |
| crc-ccitt | 31.4 | 25.7 |  2.6 |
|       md4 | 50.2 | 39.8 |  4.0 |
|       md5 | 44.2 | 34.9 |  3.5 |
|      sha1 | 46.7 | 37.0 |  3.7 |
|    sha256 | 46.9 | 36.5 |  3.6 |
+-----------+------+------+------+

For 512-way ECMP with 3 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.8 | 25.4 |  2.5 |
|      mask | 31.7 | 25.5 |  2.5 |
|       xor | 31.5 | 25.2 |  2.5 |
| crc-ccitt | 32.1 | 25.5 |  2.6 |
|       md4 | 39.8 | 32.2 |  3.2 |
|       md5 | 38.7 | 30.8 |  3.1 |
|      sha1 | 38.6 | 31.4 |  3.1 |
|    sha256 | 39.9 | 31.2 |  3.1 |
+-----------+------+------+------+

For 1024-way ECMP with 3 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 33.1 | 26.5 |  2.6 |
|      mask | 32.7 | 26.3 |  2.6 |
|       xor | 31.5 | 25.1 |  2.5 |
| crc-ccitt | 31.9 | 25.5 |  2.5 |
|       md4 | 35.5 | 28.1 |  2.8 |
|       md5 | 36.9 | 29.2 |  2.9 |
|      sha1 | 35.7 | 28.6 |  2.9 |
|    sha256 | 35.8 | 28.5 |  2.8 |
+-----------+------+------+------+

For 2048-way ECMP with 3 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.3 | 24.9 |  2.5 |
|      mask | 31.5 | 24.9 |  2.5 |
|       xor | 32.3 | 25.7 |  2.6 |
| crc-ccitt | 30.9 | 24.4 |  2.4 |
|       md4 | 33.7 | 26.6 |  2.7 |
|       md5 | 33.8 | 27.3 |  2.7 |
|      sha1 | 32.6 | 26.4 |  2.6 |
|    sha256 | 33.5 | 26.6 |  2.7 |
+-----------+------+------+------+

For 4096-way ECMP with 3 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 32.2 | 25.5 |  2.5 |
|      mask | 31.3 | 25.1 |  2.5 |
|       xor | 31.8 | 25.2 |  2.5 |
| crc-ccitt | 31.7 | 25.1 |  2.5 |
|       md4 | 33.3 | 26.4 |  2.6 |
|       md5 | 32.8 | 26.2 |  2.6 |
|      sha1 | 32.9 | 26.3 |  2.6 |
|    sha256 | 32.8 | 26.2 |  2.6 |
+-----------+------+------+------+

For 2-way ECMP with 4 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  24.0 |  24.0 |  2.4 |
|      mask |   5.0 |   5.0 |  0.5 |
|       xor |  26.0 |  26.0 |  2.6 |
| crc-ccitt |  26.0 |  26.0 |  2.6 |
|       md4 |  46.0 |  46.0 |  4.6 |
|       md5 | 250.0 | 250.0 | 25.0 |
|      sha1 |  75.0 |  75.0 |  7.5 |
|    sha256 | 262.0 | 262.0 | 26.2 |
+-----------+-------+-------+------+

For 4-way ECMP with 4 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  39.3 |  32.0 |  3.2 |
|      mask |  17.7 |  13.5 |  1.4 |
|       xor |  28.4 |  22.5 |  2.2 |
| crc-ccitt |  29.5 |  22.0 |  2.2 |
|       md4 | 180.0 | 151.5 | 15.2 |
|       md5 | 148.5 | 130.0 | 13.0 |
|      sha1 | 116.0 | 100.0 | 10.0 |
|    sha256 | 128.4 | 113.5 | 11.3 |
+-----------+-------+-------+------+

For 8-way ECMP with 4 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  32.7 |  27.0 |  2.7 |
|      mask |  32.7 |  25.2 |  2.5 |
|       xor |  37.7 |  26.2 |  2.6 |
| crc-ccitt |  30.3 |  23.2 |  2.3 |
|       md4 |  79.2 |  68.5 |  6.9 |
|       md5 | 148.2 | 133.0 | 13.3 |
|      sha1 | 112.1 | 101.8 | 10.2 |
|    sha256 | 120.7 | 108.8 | 10.9 |
+-----------+-------+-------+------+

For 16-way ECMP with 4 more bits of entropy:
+-----------+-------+------+------+
|      Hash |  RMSE |  MAE | MAPE |
+-----------+-------+------+------+
|       add |  26.7 | 21.0 |  2.1 |
|      mask |  42.9 | 34.9 |  3.5 |
|       xor |  31.5 | 24.0 |  2.4 |
| crc-ccitt |  33.9 | 29.1 |  2.9 |
|       md4 | 120.1 | 94.6 |  9.5 |
|       md5 | 106.8 | 86.4 |  8.6 |
|      sha1 |  90.1 | 71.1 |  7.1 |
|    sha256 |  65.7 | 58.9 |  5.9 |
+-----------+-------+------+------+

For 32-way ECMP with 4 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 33.2 | 25.9 |  2.6 |
|      mask | 30.3 | 24.7 |  2.5 |
|       xor | 38.8 | 32.2 |  3.2 |
| crc-ccitt | 28.0 | 21.9 |  2.2 |
|       md4 | 63.9 | 48.1 |  4.8 |
|       md5 | 79.4 | 61.7 |  6.2 |
|      sha1 | 63.1 | 49.9 |  5.0 |
|    sha256 | 83.0 | 69.1 |  6.9 |
+-----------+------+------+------+

For 64-way ECMP with 4 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 28.7 | 22.3 |  2.2 |
|      mask | 30.2 | 25.5 |  2.6 |
|       xor | 33.2 | 26.8 |  2.7 |
| crc-ccitt | 30.2 | 23.3 |  2.3 |
|       md4 | 52.5 | 42.5 |  4.2 |
|       md5 | 62.2 | 51.2 |  5.1 |
|      sha1 | 55.1 | 45.0 |  4.5 |
|    sha256 | 52.0 | 42.3 |  4.2 |
+-----------+------+------+------+

For 128-way ECMP with 4 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.6 | 25.0 |  2.5 |
|      mask | 29.0 | 21.8 |  2.2 |
|       xor | 30.1 | 23.0 |  2.3 |
| crc-ccitt | 29.7 | 24.4 |  2.4 |
|       md4 | 46.5 | 36.9 |  3.7 |
|       md5 | 46.1 | 36.6 |  3.7 |
|      sha1 | 42.9 | 33.8 |  3.4 |
|    sha256 | 50.0 | 40.4 |  4.0 |
+-----------+------+------+------+

For 256-way ECMP with 4 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.3 | 24.9 |  2.5 |
|      mask | 30.5 | 23.8 |  2.4 |
|       xor | 31.0 | 25.4 |  2.5 |
| crc-ccitt | 32.7 | 26.1 |  2.6 |
|       md4 | 40.8 | 32.7 |  3.3 |
|       md5 | 41.5 | 33.0 |  3.3 |
|      sha1 | 36.4 | 28.5 |  2.8 |
|    sha256 | 38.4 | 30.1 |  3.0 |
+-----------+------+------+------+

For 512-way ECMP with 4 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.4 | 25.3 |  2.5 |
|      mask | 32.5 | 26.0 |  2.6 |
|       xor | 31.5 | 24.7 |  2.5 |
| crc-ccitt | 31.6 | 25.1 |  2.5 |
|       md4 | 34.3 | 27.3 |  2.7 |
|       md5 | 36.1 | 29.2 |  2.9 |
|      sha1 | 35.9 | 28.6 |  2.9 |
|    sha256 | 37.1 | 29.5 |  2.9 |
+-----------+------+------+------+

For 1024-way ECMP with 4 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.3 | 24.8 |  2.5 |
|      mask | 31.7 | 25.2 |  2.5 |
|       xor | 31.4 | 25.1 |  2.5 |
| crc-ccitt | 31.5 | 25.3 |  2.5 |
|       md4 | 33.1 | 26.3 |  2.6 |
|       md5 | 34.1 | 27.0 |  2.7 |
|      sha1 | 33.7 | 27.1 |  2.7 |
|    sha256 | 32.8 | 26.1 |  2.6 |
+-----------+------+------+------+

For 2048-way ECMP with 4 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.9 | 25.4 |  2.5 |
|      mask | 31.8 | 25.5 |  2.5 |
|       xor | 32.2 | 25.7 |  2.6 |
| crc-ccitt | 31.4 | 24.8 |  2.5 |
|       md4 | 32.7 | 26.1 |  2.6 |
|       md5 | 32.9 | 26.1 |  2.6 |
|      sha1 | 33.0 | 26.3 |  2.6 |
|    sha256 | 31.9 | 25.7 |  2.6 |
+-----------+------+------+------+

For 4096-way ECMP with 4 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.7 | 25.4 |  2.5 |
|      mask | 32.5 | 25.8 |  2.6 |
|       xor | 31.7 | 25.2 |  2.5 |
| crc-ccitt | 32.0 | 25.7 |  2.6 |
|       md4 | 32.6 | 26.0 |  2.6 |
|       md5 | 31.6 | 25.1 |  2.5 |
|      sha1 | 31.7 | 25.1 |  2.5 |
|    sha256 | 32.3 | 25.9 |  2.6 |
+-----------+------+------+------+

For 2-way ECMP with 5 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  29.0 |  29.0 |  2.9 |
|      mask |  11.0 |  11.0 |  1.1 |
|       xor |   7.0 |   7.0 |  0.7 |
| crc-ccitt |   7.0 |   7.0 |  0.7 |
|       md4 | 358.0 | 358.0 | 35.8 |
|       md5 | 174.0 | 174.0 | 17.4 |
|      sha1 | 108.0 | 108.0 | 10.8 |
|    sha256 |  10.0 |  10.0 |  1.0 |
+-----------+-------+-------+------+

For 4-way ECMP with 5 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  41.3 |  35.0 |  3.5 |
|      mask |  42.7 |  33.5 |  3.4 |
|       xor |  17.5 |  15.5 |  1.6 |
| crc-ccitt |  14.2 |  12.5 |  1.2 |
|       md4 | 135.3 | 125.0 | 12.5 |
|       md5 | 128.0 | 109.0 | 10.9 |
|      sha1 | 114.3 | 106.5 | 10.7 |
|    sha256 | 175.5 | 141.5 | 14.2 |
+-----------+-------+-------+------+

For 8-way ECMP with 5 more bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|       add |  23.5 |  20.0 |  2.0 |
|      mask |  29.0 |  20.2 |  2.0 |
|       xor |  27.1 |  24.5 |  2.4 |
| crc-ccitt |  27.6 |  20.2 |  2.0 |
|       md4 | 105.4 |  91.5 |  9.2 |
|       md5 |  53.3 |  43.5 |  4.3 |
|      sha1 |  74.8 |  63.8 |  6.4 |
|    sha256 | 124.7 | 102.2 | 10.2 |
+-----------+-------+-------+------+

For 16-way ECMP with 5 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 39.1 | 33.2 |  3.3 |
|      mask | 38.5 | 29.5 |  2.9 |
|       xor | 41.3 | 33.6 |  3.4 |
| crc-ccitt | 27.0 | 21.2 |  2.1 |
|       md4 | 71.6 | 59.1 |  5.9 |
|       md5 | 86.5 | 73.8 |  7.4 |
|      sha1 | 79.3 | 63.4 |  6.3 |
|    sha256 | 59.3 | 50.9 |  5.1 |
+-----------+------+------+------+

For 32-way ECMP with 5 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.2 | 25.1 |  2.5 |
|      mask | 33.7 | 28.6 |  2.9 |
|       xor | 32.6 | 27.6 |  2.8 |
| crc-ccitt | 28.7 | 22.0 |  2.2 |
|       md4 | 51.4 | 42.0 |  4.2 |
|       md5 | 54.2 | 44.8 |  4.5 |
|      sha1 | 68.5 | 49.2 |  4.9 |
|    sha256 | 56.7 | 47.8 |  4.8 |
+-----------+------+------+------+

For 64-way ECMP with 5 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 33.0 | 26.6 |  2.7 |
|      mask | 29.4 | 22.9 |  2.3 |
|       xor | 33.9 | 27.1 |  2.7 |
| crc-ccitt | 29.6 | 22.0 |  2.2 |
|       md4 | 46.6 | 35.8 |  3.6 |
|       md5 | 48.5 | 39.8 |  4.0 |
|      sha1 | 43.4 | 33.8 |  3.4 |
|    sha256 | 44.4 | 36.2 |  3.6 |
+-----------+------+------+------+

For 128-way ECMP with 5 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 32.3 | 25.7 |  2.6 |
|      mask | 28.8 | 22.9 |  2.3 |
|       xor | 30.0 | 24.5 |  2.5 |
| crc-ccitt | 31.3 | 23.9 |  2.4 |
|       md4 | 41.7 | 34.7 |  3.5 |
|       md5 | 38.5 | 31.1 |  3.1 |
|      sha1 | 36.4 | 28.8 |  2.9 |
|    sha256 | 35.0 | 28.0 |  2.8 |
+-----------+------+------+------+

For 256-way ECMP with 5 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 30.8 | 24.4 |  2.4 |
|      mask | 31.0 | 24.6 |  2.5 |
|       xor | 31.3 | 24.7 |  2.5 |
| crc-ccitt | 31.1 | 24.3 |  2.4 |
|       md4 | 37.0 | 29.6 |  3.0 |
|       md5 | 31.5 | 26.0 |  2.6 |
|      sha1 | 35.5 | 27.7 |  2.8 |
|    sha256 | 37.0 | 29.6 |  3.0 |
+-----------+------+------+------+

For 512-way ECMP with 5 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 32.0 | 25.2 |  2.5 |
|      mask | 30.9 | 24.7 |  2.5 |
|       xor | 30.3 | 24.3 |  2.4 |
| crc-ccitt | 31.6 | 25.0 |  2.5 |
|       md4 | 36.0 | 28.6 |  2.9 |
|       md5 | 33.9 | 26.5 |  2.6 |
|      sha1 | 32.7 | 25.9 |  2.6 |
|    sha256 | 33.7 | 26.5 |  2.6 |
+-----------+------+------+------+

For 1024-way ECMP with 5 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 32.5 | 25.7 |  2.6 |
|      mask | 32.1 | 25.7 |  2.6 |
|       xor | 32.4 | 26.1 |  2.6 |
| crc-ccitt | 31.2 | 25.5 |  2.5 |
|       md4 | 33.4 | 26.8 |  2.7 |
|       md5 | 32.2 | 25.9 |  2.6 |
|      sha1 | 33.4 | 26.3 |  2.6 |
|    sha256 | 33.3 | 26.5 |  2.7 |
+-----------+------+------+------+

For 2048-way ECMP with 5 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 30.9 | 24.8 |  2.5 |
|      mask | 32.1 | 25.9 |  2.6 |
|       xor | 31.9 | 25.5 |  2.5 |
| crc-ccitt | 31.3 | 24.9 |  2.5 |
|       md4 | 32.3 | 25.8 |  2.6 |
|       md5 | 32.4 | 25.9 |  2.6 |
|      sha1 | 32.6 | 26.1 |  2.6 |
|    sha256 | 32.6 | 25.9 |  2.6 |
+-----------+------+------+------+

For 4096-way ECMP with 5 more bits of entropy:
+-----------+------+------+------+
|      Hash | RMSE |  MAE | MAPE |
+-----------+------+------+------+
|       add | 31.8 | 25.2 |  2.5 |
|      mask | 31.5 | 24.9 |  2.5 |
|       xor | 31.6 | 25.1 |  2.5 |
| crc-ccitt | 31.6 | 25.3 |  2.5 |
|       md4 | 31.9 | 25.5 |  2.6 |
|       md5 | 31.8 | 25.2 |  2.5 |
|      sha1 | 32.2 | 25.7 |  2.6 |
|    sha256 | 32.2 | 26.0 |  2.6 |
+-----------+------+------+------+

Error per added bit
+------------+-------+-------+-------+
| Added bits | MRMSE |  MMAE | MMAPE |
+------------+-------+-------+-------+
|          0 | 171.0 | 150.6 |  15.1 |
|          1 | 115.8 | 100.2 |  10.0 |
|          2 |  74.9 |  61.6 |   6.2 |
|          3 |  58.5 |  49.0 |   4.9 |
|          4 |  51.1 |  43.2 |   4.3 |
|          5 |  47.1 |  39.8 |   4.0 |
+------------+-------+-------+-------+

Error per algorithm
+-----------+-------+-------+-------+
| Algorithm | MRMSE |  MMAE | MMAPE |
+-----------+-------+-------+-------+
|       add |  52.7 |  45.8 |   4.6 |
|      mask |  34.1 |  28.6 |   2.9 |
|       xor |  54.0 |  48.0 |   4.8 |
| crc-ccitt |  48.7 |  43.2 |   4.3 |
|       md4 | 137.7 | 119.8 |  12.0 |
|       md5 | 131.3 | 110.7 |  11.1 |
|      sha1 | 110.5 |  93.0 |   9.3 |
|    sha256 | 122.2 | 103.3 |  10.3 |
+-----------+-------+-------+-------+

For 3597-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  31.7 |  25.2 |  2.5 |
| crc-ccitt | 264.8 | 210.5 | 21.0 |
|       md4 | 356.1 | 283.5 | 28.3 |
|       md5 | 357.9 | 286.4 | 28.6 |
|      sha1 | 358.9 | 284.9 | 28.5 |
|    sha256 | 357.1 | 283.1 | 28.3 |
+-----------+-------+-------+------+

For 911-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  31.5 |  25.4 |  2.5 |
| crc-ccitt | 163.2 | 133.3 | 13.3 |
|       md4 | 181.9 | 144.9 | 14.5 |
|       md5 | 190.0 | 150.5 | 15.0 |
|      sha1 | 181.6 | 147.2 | 14.7 |
|    sha256 | 180.3 | 143.8 | 14.4 |
+-----------+-------+-------+------+

For 3023-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  31.8 |  25.6 |  2.6 |
| crc-ccitt | 238.2 | 191.1 | 19.1 |
|       md4 | 320.7 | 254.9 | 25.5 |
|       md5 | 319.7 | 253.8 | 25.4 |
|      sha1 | 329.1 | 262.8 | 26.3 |
|    sha256 | 328.0 | 261.6 | 26.2 |
+-----------+-------+-------+------+

For 2288-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  31.5 |  25.2 |  2.5 |
| crc-ccitt | 190.8 | 156.0 | 15.6 |
|       md4 | 282.4 | 225.8 | 22.6 |
|       md5 | 279.8 | 220.4 | 22.0 |
|      sha1 | 280.8 | 225.2 | 22.5 |
|    sha256 | 281.9 | 223.5 | 22.3 |
+-----------+-------+-------+------+

For 2761-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  32.1 |  25.4 |  2.5 |
| crc-ccitt | 225.4 | 179.9 | 18.0 |
|       md4 | 318.4 | 253.0 | 25.3 |
|       md5 | 309.4 | 247.8 | 24.8 |
|      sha1 | 304.6 | 244.0 | 24.4 |
|    sha256 | 309.2 | 247.2 | 24.7 |
+-----------+-------+-------+------+

For 1845-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  31.4 |  25.0 |  2.5 |
| crc-ccitt | 161.2 | 129.6 | 13.0 |
|       md4 | 260.3 | 207.3 | 20.7 |
|       md5 | 257.3 | 203.1 | 20.3 |
|      sha1 | 259.9 | 206.1 | 20.6 |
|    sha256 | 245.7 | 196.3 | 19.6 |
+-----------+-------+-------+------+

For 3508-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  31.4 |  25.2 |  2.5 |
| crc-ccitt | 209.0 | 167.7 | 16.8 |
|       md4 | 352.0 | 280.6 | 28.1 |
|       md5 | 346.1 | 277.6 | 27.8 |
|      sha1 | 357.4 | 284.2 | 28.4 |
|    sha256 | 344.9 | 274.3 | 27.4 |
+-----------+-------+-------+------+

For 1963-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  31.4 |  25.3 |  2.5 |
| crc-ccitt | 163.5 | 131.9 | 13.2 |
|       md4 | 257.7 | 205.5 | 20.6 |
|       md5 | 257.2 | 204.8 | 20.5 |
|      sha1 | 264.4 | 213.0 | 21.3 |
|    sha256 | 266.7 | 214.1 | 21.4 |
+-----------+-------+-------+------+

For 1168-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  32.2 |  25.4 |  2.5 |
| crc-ccitt | 107.5 |  86.1 |  8.6 |
|       md4 | 198.6 | 158.4 | 15.8 |
|       md5 | 203.3 | 161.8 | 16.2 |
|      sha1 | 201.1 | 162.3 | 16.2 |
|    sha256 | 201.4 | 159.0 | 15.9 |
+-----------+-------+-------+------+

For 2274-way ECMP, 16 bits of entropy:
+-----------+-------+-------+------+
|      Hash |  RMSE |   MAE | MAPE |
+-----------+-------+-------+------+
|      mask |  31.7 |  25.0 |  2.5 |
| crc-ccitt | 208.9 | 166.0 | 16.6 |
|       md4 | 277.2 | 221.8 | 22.2 |
|       md5 | 279.8 | 224.9 | 22.5 |
|      sha1 | 285.5 | 225.7 | 22.6 |
|    sha256 | 288.4 | 229.6 | 23.0 |
+-----------+-------+-------+------+

For n-way ECMP, with 16 bits of entropy, across hash functions
+--------+-------+-------+-------+
| n-ways | MRMSE |  MMAE | MMAPE |
+--------+-------+-------+-------+
|   3597 | 287.8 | 228.9 |  22.9 |
|    911 | 154.8 | 124.2 |  12.4 |
|   3023 | 261.2 | 208.3 |  20.8 |
|   2288 | 224.5 | 179.3 |  17.9 |
|   2761 | 249.9 | 199.5 |  19.9 |
|   1845 | 202.6 | 161.2 |  16.1 |
|   3508 | 273.5 | 218.3 |  21.8 |
|   1963 | 206.8 | 165.8 |  16.6 |
|   1168 | 157.3 | 125.5 |  12.5 |
|   2274 | 228.6 | 182.2 |  18.2 |
+--------+-------+-------+-------+

Author's Address

Tony Li
Juniper Networks