# Hardware Checklist

LLMS index: [llms.txt](/llms.txt)

---

<a id="hardware-checklist"></a>
<a id="minio-hardware-checklist"></a>

Use the following checklist when planning the hardware configuration for a production, distributed MinIO deployment.

## Considerations {#considerations}

When selecting hardware for your MinIO implementation, take into account the following factors:

- Expected amount of data in tebibytes to store at launch
- Expected growth in size of data for at least the next two years
- Number of objects by average object size
- Average retention time of data in years
- Number of sites to be deployed
- Number of expected buckets

<a id="deploy-minio-distributed-recommendations"></a>

## Production Hardware Recommendations {#production-hardware-recommendations}

The following checklist follows MinIO’s [Recommended Configuration](https://min.io/product/reference-hardware?ref-docs) for production deployments. The provided guidance is intended as a baseline and cannot replace [MinIO SUBNET](https://min.io/pricing?jmp=docs) Performance Diagnostics, Architecture Reviews, and direct-to-engineering support.

MinIO, like any distributed system, benefits from selecting identical configurations for all nodes in a given [server pool](/glossary/#term-server-pool). Ensure a consistent selection of hardware (CPU, memory, motherboard, storage adapters) and software (operating system, kernel settings, system services) across pool nodes.

Deployments may exhibit unpredictable performance if nodes have varying hardware or software configurations. Workloads that benefit from storing aged data on lower-cost hardware should instead deploy a dedicated “warm” or “cold” MinIO deployment and [transition](/administration/object-management/object-lifecycle-management/#minio-lifecycle-management-tiering) data to that tier.

> [!NOTE]
> **MinIO does not provide hosted services or hardware sales**
>
> See our [Reference Hardware](https://min.io/product/reference-hardware#hardware?ref-docs) page for a curated selection of servers and storage components from our hardware partners.

**Kubernetes**

<table>
  <thead>
    <tr>
      <th></th>
      <th><p>Description</p></th>
      <th><p>Minimum</p></th>
      <th><p>Recommended</p></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><p><svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-circle" viewBox="0 0 16 16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1.5a6.5 6.5 0 100 13 6.5 6.5 0 000-13zM0 8a8 8 0 1116 0A8 8 0 010 8z"></path></svg></p></td>
      <td><p>Kubernetes worker nodes to exclusively service the MinIO Tenant.</p></td>
      <td><p>4 workers per Tenant</p></td>
      <td><p>8+ workers per Tenant</p></td>
    </tr>
    <tr>
      <td><p><svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-circle" viewBox="0 0 16 16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1.5a6.5 6.5 0 100 13 6.5 6.5 0 000-13zM0 8a8 8 0 1116 0A8 8 0 010 8z"></path></svg></p></td>
      <td><p><a href="#minio-hardware-checklist-storage">Dedicated Persistent Volumes for the MinIO Tenant</a>.</p></td>
      <td><p>4 PV per MinIO Server pod</p></td>
      <td><p>8+ PV per MinIO Server pod</p></td>
    </tr>
    <tr>
      <td><p><svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-circle" viewBox="0 0 16 16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1.5a6.5 6.5 0 100 13 6.5 6.5 0 000-13zM0 8a8 8 0 1116 0A8 8 0 010 8z"></path></svg></p></td>
      <td><p><a href="#minio-hardware-checklist-network">High speed network infrastructure</a>.</p></td>
      <td><p>25GbE</p></td>
      <td><p>100GbE</p></td>
    </tr>
    <tr>
      <td><p><svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-circle" viewBox="0 0 16 16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1.5a6.5 6.5 0 100 13 6.5 6.5 0 000-13zM0 8a8 8 0 1116 0A8 8 0 010 8z"></path></svg></p></td>
      <td><p>Server-grade CPUs with support for modern SIMD instructions (AVX-512), such as Intel® Xeon® Scalable or better.</p></td>
      <td><p>4 vCPU per MinIO Pod</p></td>
      <td><p>8+ vCPU per MinIO Pod</p></td>
    </tr>
    <tr>
      <td><p><svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-circle" viewBox="0 0 16 16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1.5a6.5 6.5 0 100 13 6.5 6.5 0 000-13zM0 8a8 8 0 1116 0A8 8 0 010 8z"></path></svg></p></td>
      <td><p><a href="#minio-hardware-checklist-memory">Available memory to meet or exceed per-server usage</a> by a reasonable buffer.</p></td>
      <td><p>32GB of available memory per worker node</p></td>
      <td><p>128GB+ of available memory per worker node</p></td>
    </tr>
  </tbody>
</table>

**Baremetal**

<table>
  <thead>
    <tr>
      <th></th>
      <th><p>Description</p></th>
      <th><p>Minimum</p></th>
      <th><p>Recommended</p></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><p><svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-circle" viewBox="0 0 16 16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1.5a6.5 6.5 0 100 13 6.5 6.5 0 000-13zM0 8a8 8 0 1116 0A8 8 0 010 8z"></path></svg></p></td>
      <td><p>Dedicated Baremetal or Virtual Hosts (“hosts”).</p></td>
      <td><p>4 dedicated hosts</p></td>
      <td><p>8+ dedicated hosts</p></td>
    </tr>
    <tr>
      <td><p><svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-circle" viewBox="0 0 16 16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1.5a6.5 6.5 0 100 13 6.5 6.5 0 000-13zM0 8a8 8 0 1116 0A8 8 0 010 8z"></path></svg></p></td>
      <td><p><a href="#minio-hardware-checklist-storage">Dedicated locally-attached drives for each host</a>.</p></td>
      <td><p>4 drives per MinIO Server</p></td>
      <td><p>8+ drives per MinIO Server</p></td>
    </tr>
    <tr>
      <td><p><svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-circle" viewBox="0 0 16 16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1.5a6.5 6.5 0 100 13 6.5 6.5 0 000-13zM0 8a8 8 0 1116 0A8 8 0 010 8z"></path></svg></p></td>
      <td><p><a href="#minio-hardware-checklist-network">High speed network infrastructure</a>.</p></td>
      <td><p>25GbE</p></td>
      <td><p>100GbE</p></td>
    </tr>
    <tr>
      <td><p><svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-circle" viewBox="0 0 16 16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1.5a6.5 6.5 0 100 13 6.5 6.5 0 000-13zM0 8a8 8 0 1116 0A8 8 0 010 8z"></path></svg></p></td>
      <td><p>Server-grade CPUs with support for modern SIMD instructions (AVX-512), such as Intel® Xeon® Scalable or better.</p></td>
      <td><p>8 CPU/socket or vCPU per host</p></td>
      <td><p>16+ CPU/socket or vCPU per host</p></td>
    </tr>
    <tr>
      <td><p><svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-circle" viewBox="0 0 16 16" aria-hidden="true"><path fill-rule="evenodd" d="M8 1.5a6.5 6.5 0 100 13 6.5 6.5 0 000-13zM0 8a8 8 0 1116 0A8 8 0 010 8z"></path></svg></p></td>
      <td><p><a href="#minio-hardware-checklist-memory">Available memory to meet or exceed per-server usage</a> by a reasonable buffer.</p></td>
      <td><p>32GB of available memory per host</p></td>
      <td><p>128GB+ of available memory per host</p></td>
    </tr>
  </tbody>
</table>

> [!WARNING]
> **Important**
>
> The following areas have the greatest impact on MinIO performance, listed in order of importance:
>
> <table>
>   <tbody>
>     <tr>
>       <td><p>Network Infrastructure</p></td>
>       <td><p>Insufficient or limited throughput constrains performance</p></td>
>     </tr>
>     <tr>
>       <td><p>Storage Controller</p></td>
>       <td><p>Old firmware, limited throughput, or failing hardware constrains performance and affects reliability</p></td>
>     </tr>
>     <tr>
>       <td><p>Storage (Drive)</p></td>
>       <td><p>Old firmware, or slow/aging/failing hardware constrains performance and affects reliability</p></td>
>     </tr>
>   </tbody>
> </table>
>
> Prioritize securing the necessary components for each of these areas before focusing on other hardware resources, such as compute-related constraints.

The minimum recommendations above reflect MinIO’s experience with assisting enterprise customers in deploying on a variety of IT infrastructures while maintaining the desired SLA/SLO. While MinIO may run on less than the minimum recommended topology, any potential cost savings come at the risk of decreased reliability, performance, or overall functionality.

<a id="minio-hardware-checklist-network"></a>

### Networking {#networking}

MinIO recommends high speed networking to support the maximum possible throughput of the attached storage (aggregated drives, storage controllers, and PCIe busses). The following table provides a general guideline for the maximum storage throughput supported by a given physical or virtual network interface. This table assumes all network infrastructure components, such as routers, switches, and physical cabling, also supports the NIC bandwidth.

<table>
  <tbody>
    <tr>
      <td><p>NIC Bandwidth (Gbps)</p></td>
      <td><p>Estimated Aggregated Storage Throughput (GBps)</p></td>
    </tr>
    <tr>
      <td><p>10Gbps</p></td>
      <td><p>1.25GBps</p></td>
    </tr>
    <tr>
      <td><p>25Gbps</p></td>
      <td><p>3.125GBps</p></td>
    </tr>
    <tr>
      <td><p>50Gbps</p></td>
      <td><p>6.25GBps</p></td>
    </tr>
    <tr>
      <td><p>100Gbps</p></td>
      <td><p>12.5GBps</p></td>
    </tr>
  </tbody>
</table>

Networking has the greatest impact on MinIO performance, where low per-host bandwidth artificially constrains the potential performance of the storage. The following examples of network throughput constraints assume spinning disks with ~100MB/S sustained I/O

- 1GbE network link can support up to 125MB/s, or one spinning disk
- 10GbE network can support approximately 1.25GB/s, potentially supporting 10-12 spinning disks
- 25GbE network can support approximately 3.125GB/s, potentially supporting ~30 spinning disks

<a id="minio-hardware-checklist-memory"></a>

### Memory {#memory}

Memory primarily constrains the number of concurrent simultaneous connections per node.

You can calculate the maximum number of concurrent requests per node with this formula:

> \(totalRam / ramPerRequest\)

To calculate the amount of RAM used for each request, use this formula:

> \(((2MiB + 128KiB) \times driveCount) + (2 \times 10MiB) + (2 \times 1MiB)\)
>
> 10MiB is the default erasure block size v1. 1 MiB is the default erasure block size v2.

The following table lists the maximum concurrent requests on a node based on the number of host drives and the *free* system RAM:

| Number of Drives | 32 GiB of RAM | 64 GiB of RAM | 128 GiB of RAM | 256 GiB of RAM | 512 GiB of RAM |
| --- | --- | --- | --- | --- | --- |
| 4 Drives | 1,074 | 2,149 | 4,297 | 8,595 | 17,190 |
| 8 Drives | 840 | 1,680 | 3,361 | 6,722 | 13,443 |
| 16 Drives | 585 | 1,170 | 2.341 | 4,681 | 9,362 |

The following table provides general guidelines for allocating memory for use by MinIO based on the total amount of local storage on the node:

| Total Host Storage | Recommended Host Memory |
| --- | --- |
| Up to 1 Tebibyte (Ti) | 8GiB |
| Up to 10 Tebibyte (Ti) | 16GiB |
| Up to 100 Tebibyte (Ti) | 32GiB |
| Up to 1 Pebibyte (Pi) | 64GiB |
| More than 1 Pebibyte (Pi) | 128GiB |

> [!WARNING]
> **Important**
>
> Starting with [RELEASE.2024-01-28T22-35-53Z](https://github.com/minio/minio/releases/tag/RELEASE.2024-01-28T22-35-53Z), MinIO preallocates 2GiB of memory per node in distributed setups and 1GiB of memory for a single-node setup.

<a id="minio-hardware-checklist-storage"></a>

### Storage {#storage}

> [!NOTE]
> **Exclusive access to drives**
>
> MinIO **requires** *exclusive* access to the drives or volumes provided for object storage. No other processes, software, scripts, or persons should perform *any* actions directly on the drives or volumes provided to MinIO or the objects or files MinIO places on them.
>
> Unless directed by MinIO Engineering, do not use scripts or tools to directly modify, delete, or move any of the data shards, parity shards, or metadata files on the provided drives, including from one drive or node to another. Such operations are very likely to result in widespread corruption and data loss beyond MinIO’s ability to heal.

#### Recommended Storage Mediums {#recommended-storage-mediums}

**Kubernetes**

MinIO recommends provisioning a storage class for each MinIO Tenant that meets the performance objectives for that tenant.

Where possible, configure the Storage Class, CSI, or other provisioner underlying the PV to format volumes as XFS to ensure best performance.

Ensure a consistent underlying storage type (NVMe, SSD, HDD) for all PVs provisioned in a Tenant.

Ensure the same presented capacity of each PV across all nodes in each Tenant [server pool](/operations/concepts/#minio-intro-server-pool). MinIO limits the maximum usable size per PV to the smallest PV in the pool. For example, if a pool has 15 10TB PVs and 1 1TB PV, MinIO limits the per-PV capacity to 1TB.

**Baremetal**

MinIO recommends using flash-based storage (NVMe or SSD) for all workload types and scales. Workloads that require high performance should prefer NVMe over SSD.

MinIO does not recommends HDD storage for production environments. HDD storage typically does not provide the necessary performance to meet the expectations of modern workloads, and any cost efficiencies at scale are offset by the performance constraints of the medium.

#### Prefer Direct-Attached “Local” Storage (DAS) {#prefer-direct-attached-local-storage-das}

<abbr title="Direct-Attached Storage">DAS</abbr>, such as locally-attached JBOD (Just a Bunch of Disks) arrays, provide significant performance and consistency advantages over networked (NAS, SAN, NFS) storage.

**Kubernetes**

While MinIO Tenants can make use of remote Persistent Volume (PV) resources, the cost of performing I/O over the network typically constrains overall performance.

MinIO strongly recommends using CSIs which can provision storage attached to the worker node on which Kubernetes schedules your MinIO pods, such as [MinIO DirectPV](https://github.com/minio/directpv).

For all other cases, make every effort possible to select a CSI which presents the storage to MinIO as if it were a locally-attached filesystem. CSIs which add layers of software or translations between MinIO and the OS-level storage access APIs necessarily increase the complexity of the syste and can contribute to unexpected or undesired behavior.

**Baremetal**

Configure the JBOD arrays without any RAID, pooling, or similar software-level layers, such that the storage is presented directly to MinIO.

For virtual machines or systems that require provising storage as a virtual volume, MinIO recommends using thick LUNs only.

> [!DETAILS]- Network File System Volumes Break Consistency Guarantees
> MinIO’s strict **read-after-write** and **list-after-write** consistency model requires local drive filesystems. MinIO cannot provide consistency guarantees if the underlying storage volumes are NFS or a similar network-attached storage volume.

#### Use XFS-Formatted Drives with Consistent Mounting {#use-xfs-formatted-drives-with-consistent-mounting}

**Kubernetes**

MinIO recommends formatting the drives underlying MinIO Persistent Volumes as `xfs`.

If using a CSI, review the documentation for that CSI and ensure it supports specifying the `xfs` filesystem. MinIO strongly recommends avoiding any CSI which formats drives as `ext4`, `btrfs` or other filesystems.

MinIO expects all provisioned Persistent Volumes (PV) to be intended for its exclusive use, where the underlying storage medium guarantees access to the stored data at the assigned mount path. Modifications to the underlying storage medium, including but not limited to external or third-party applications or the arbitrary re-mounting of locally-attached storage, may result in unexpected behavior or data loss.

**Baremetal**

Format drives as XFS and present them to MinIO as a <abbr title="Just a Bunch of Disks">JBOD</abbr> array with no RAID or other pooling configurations. Using any other type of backing storage (SAN/NAS, ext4, RAID, LVM) typically results in a reduction in performance, reliability, predictability, and consistency.

When formatting XFS drives, apply a unique label per drive. For example, the following command formats four drives as XFS and applies a corresponding drive label.

```shell
mkfs.xfs /dev/sdb -L MINIODRIVE1
mkfs.xfs /dev/sdc -L MINIODRIVE2
mkfs.xfs /dev/sdd -L MINIODRIVE3
mkfs.xfs /dev/sde -L MINIODRIVE4
```

MinIO **requires** that drives maintain their ordering at the mounted position across restarts. MinIO **does not** support arbitrary migration of a drive with existing MinIO data to a new mount position, whether intentional or as the result of OS-level behavior.

You **must** use `/etc/fstab` or a similar mount control system to mount drives at a consistent path. For example:

```shell
$ nano /etc/fstab

# <file system>        <mount point>    <type>  <options>         <dump>  <pass>
LABEL=MINIODRIVE1      /mnt/drive-1     xfs     defaults,noatime  0       2
LABEL=MINIODRIVE2      /mnt/drive-2     xfs     defaults,noatime  0       2
LABEL=MINIODRIVE3      /mnt/drive-3     xfs     defaults,noatime  0       2
LABEL=MINIODRIVE4      /mnt/drive-4     xfs     defaults,noatime  0       2
```

You can use `mount -a` to mount those drives at those paths during initial setup. The Operating System should otherwise mount these drives as part of the node startup process.

MinIO **strongly recommends** using label-based mounting rules over UUID-based rules. Label-based rules allow swapping an unhealthy or non-working drive with a replacement that has matching format and label. UUID-based rules require editing the `/etc/fstab` file to replace mappings with the new drive UUID.

> [!NOTE]
> **Note**
>
> Cloud environment instances which depend on mounted external storage may encounter boot failure if one or more of the remote file mounts return errors or failure. For example, an AWS ECS instance with mounted persistent EBS volumes may not boot with the standard `/etc/fstab` configuration if one or more EBS volumes fail to mount.
>
> You can set the `nofail` option to silence error reporting at boot and allow the instance to boot with one or more mount issues.
>
> You should not use this option on systems with locally attached disks, as silencing drive errors prevents both MinIO and the OS from responding to those errors in a normal fashion.

#### Disable XFS Retry On Error {#disable-xfs-retry-on-error}

MinIO **strongly recommends** disabling [retry-on-error](https://docs.kernel.org/admin-guide/xfs.html?highlight=xfs#error-handling) behavior using the `max_retries` configuration for the following error classes:

- `EIO` Error when reading or writing
- `ENOSPC` Error no space left on device
- `default` All other errors

The default `max_retries` setting typically directs the filesystem to retry-on-error indefinitely instead of propagating the error. MinIO can handle XFS errors appropriately, such that the retry-on-error behavior introduces at most unnecessary latency or performance degradation.

**Kubernetes**

Defer to the documentation for your preferred CSI or StorageClass on options for configuring filesystem-level settings.

**Baremetal**

The following script iterates through all drives at the specified mount path and sets the XFS `max_retries` setting to `0` or “fail immediately on error” for the recommended error classes. The script ignores any drives not mounted, either manually or through `/etc/fstab`. Modify the `/mnt/drive` line to match the pattern used for your MinIO drives.

```bash
#!/bin/bash

for i in $(df -h | grep /mnt/drive | awk '{ print $1 }'); do
      mountPath="$(df -h | grep $i | awk '{ print $6 }')"
      deviceName="$(basename $i)"
      echo "Modifying xfs max_retries and retry_timeout_seconds for drive $i mounted at $mountPath"
      echo 0 > /sys/fs/xfs/$deviceName/error/metadata/EIO/max_retries
      echo 0 > /sys/fs/xfs/$deviceName/error/metadata/ENOSPC/max_retries
      echo 0 > /sys/fs/xfs/$deviceName/error/metadata/default/max_retries
done
exit 0
```

You must run this script on all MinIO nodes and configure the script to re-run on reboot, as Linux Operating Systems do not typically persist these changes. You can use a `cron` job with the `@reboot` timing to run the above script whenever the node restarts and ensure all drives have retry-on-error disabled. Use `crontab -e` to create the following job, modifying the script path to match that on each node:

```shell
@reboot /opt/minio/xfs-retry-settings.sh
```

#### Use Consistent Drive Type and Capacity {#use-consistent-drive-type-and-capacity}

Ensure a consistent drive type (NVMe, SSD, HDD) for the underlying storage in a MinIO deployment. MinIO does not distinguish between storage types and does not support configuring “hot” or “warm” drives within a single deployment. Mixing drive types typically results in performance degradation, as the slowest drives in the deployment become a bottleneck regardless of the capabilities of the faster drives.

Use the same capacity and type of drive across all nodes in each MinIO [server pool](/operations/concepts/#minio-intro-server-pool). MinIO limits the maximum usable size per drive to the smallest size in the deployment. For example, if a deployment has 15 10TB drives and 1 1TB drive, MinIO limits the per-drive capacity to 1TB.

## Recommended Hardware Tests {#recommended-hardware-tests}

### Operating System Diagnostic Tools {#operating-system-diagnostic-tools}

If you cannot run the [`mc support diag`](/reference/minio-mc/mc-support-diag/#command-mc.support.diag) or the results show unexpected results, you can use the operating system’s default tools.

Test each drive independently on all servers to ensure they are identical in performance. Use the results of these OS-level tools to verify the capabilities of your storage hardware. Record the results for later reference.

1. Test the drive’s performance during write operations

   This tests checks a drive’s ability to write new data (uncached) to the drive by creating a specified number of blocks at up to a certain number of bytes at a time to mimic how a drive would function with writing uncached data. This allows you to see the actual drive performance with consistent file I/O.

   ```text
   dd if=/dev/zero of=/mnt/driveN/testfile bs=128k count=80000 oflag=direct conv=fdatasync > dd-write-drive1.txt
   ```

   Replace `driveN` with the path for the drive you are testing.

   <table>
     <tbody>
       <tr>
         <td><p><code>dd</code></p></td>
         <td><p>The command to copy and paste data.</p></td>
       </tr>
       <tr>
         <td><p><code>if=/dev/zero</code></p></td>
         <td><p>Read from <code>/dev/zero</code>, an system-generated endless stream of 0 bytes used to create a file of a specified size</p></td>
       </tr>
       <tr>
         <td><p><code>of=/mnt/driveN/testfile</code></p></td>
         <td><p>Write to <code>/mnt/driveN/testfile</code></p></td>
       </tr>
       <tr>
         <td><p><code>bs=128k</code></p></td>
         <td><p>Write up to 128,000 bytes at a time</p></td>
       </tr>
       <tr>
         <td><p><code>count=80000</code></p></td>
         <td><p>Write up to 80000 blocks of data</p></td>
       </tr>
       <tr>
         <td><p><code>oflag=direct</code></p></td>
         <td><p>Use direct I/O to write to avoid data from caching</p></td>
       </tr>
       <tr>
         <td><p><code>conv=fdatasync</code></p></td>
         <td><p>Physically write output file data before finishing</p></td>
       </tr>
       <tr>
         <td><p><code>&gt; dd-write-drive1.txt</code></p></td>
         <td><p>Write the contents of the operation’s output to <code>dd-write-drive1.txt</code> in the current working directory</p></td>
       </tr>
     </tbody>
   </table>

   The operation returns the number of files written, total size written in bytes, the total length of time for the operation (in seconds), and the speed of the writing in some order of bytes per second.
2. Test the drive’s performance during read operations

   ```text
   dd if=/mnt/driveN/testfile of=/dev/null bs=128k iflag=direct > dd-read-drive1.txt
   ```

   Replace `driveN` with the path for the drive you are testing.

   <table>
     <tbody>
       <tr>
         <td><p><code>dd</code></p></td>
         <td><p>The command to copy and paste data</p></td>
       </tr>
       <tr>
         <td><p><code>if=/mnt/driveN/testfile</code></p></td>
         <td><p>Read from <code>/mnt/driveN/testfile</code>; replace with the path to the file to use for testing the drive’s read performance</p></td>
       </tr>
       <tr>
         <td><p><code>of=/dev/null</code></p></td>
         <td><p>Write to <code>/dev/null</code>, a virtual file that does not persist after the operation completes</p></td>
       </tr>
       <tr>
         <td><p><code>bs=128k</code></p></td>
         <td><p>Write up to 128,000 bytes at a time</p></td>
       </tr>
       <tr>
         <td><p><code>count=80000</code></p></td>
         <td><p>Write up to 80000 blocks of data</p></td>
       </tr>
       <tr>
         <td><p><code>iflag=direct</code></p></td>
         <td><p>Use direct I/O to read and avoid data from caching</p></td>
       </tr>
       <tr>
         <td><p><code>&gt; dd-read-drive1.txt</code></p></td>
         <td><p>Write the contents of the operation’s output to <code>dd-read-drive1.txt</code> in the current working directory</p></td>
       </tr>
     </tbody>
   </table>

   Use a sufficiently sized file that mimics the primary use case for your deployment to get accurate read test results.

   The following guidelines may help during performance testing:

   - Small files: &lt; 128KB
   - Normal files: 128KB – 1GB
   - Large files: &gt; 1GB

   You can use the `head` command to create a file to use. The following command example creates a 10 Gigabyte file called `testfile`.

   ```shell
   head -c 10G </dev/urandom > testfile
   ```

   The operation returns the number of files read, total size read in bytes, the total length of time for the operation (in seconds), and the speed of the reading in bytes per second.

### Third Party Diagnostic Tools {#third-party-diagnostic-tools}

IO Controller test

Use [IOzone](http://iozone.org/) to test the input/output controller and all drives in combination. Document the performance numbers for each server in your deployment.

```shell
iozone -s 1g -r 4m -i 0 -i 1 -i 2 -I -t 160 -F /mnt/sdb1/tmpfile.{1..16} /mnt/sdc1/tmpfile.{1..16} /mnt/sdd1/tmpfile.{1..16} /mnt/sde1/tmpfile.{1..16} /mnt/sdf1/tmpfile.{1..16} /mnt/sdg1/tmpfile.{1..16} /mnt/sdh1/tmpfile.{1..16} /mnt/sdi1/tmpfile.{1..16} /mnt/sdj1/tmpfile.{1..16} /mnt/sdk1/tmpfile.{1..16} > iozone.txt
```

<table>
  <tbody>
    <tr>
      <td><p><code>-s 1g</code></p></td>
      <td><p>Size of 1G per file</p></td>
    </tr>
    <tr>
      <td><p><code>-r</code></p></td>
      <td><p>4m  4MB block size</p></td>
    </tr>
    <tr>
      <td><p><code>-i #</code></p></td>
      <td><p>0=write/rewrite, 1=read/re-read, 2=random-read/write</p></td>
    </tr>
    <tr>
      <td><p><code>-I</code></p></td>
      <td><p>Direct-IO modern</p></td>
    </tr>
    <tr>
      <td><p><code>-t N</code></p></td>
      <td><p>Number of threads (numberOfDrives * 16)</p></td>
    </tr>
    <tr>
      <td><p><code>-F &lt;&gt;</code></p></td>
      <td><p>list of files (the above command tests with 16 files per drive)</p></td>
    </tr>
  </tbody>
</table>

## Recommended tools for MinIO subscriptions {#recommended-tools-for-minio-subscriptions}

> [!WARNING]
> **Important**
>
> The tools noted in this section **require** a MinIO subscription. MinIO strongly recommends all production deployments use [AIStor Object Store](https://www.min.io/product/aistor/object-data-store) with their SUBNET license. For more information, see the [MinIO AIStor pricing page](https://min.io/pricing?jmp=docs).

1. Health diagnostic tool

   Generate a summary of the health status of your deployment. If you have access to SUBNET, you can upload the results there.

   ```shell
   mc support diag ALIAS --airgap
   ```

   Replace ALIAS with the [`alias`](/reference/minio-mc/mc-alias/#command-mc.alias) defined for the deployment.
2. Network test

   Run a network throughput test on a cluster with alias `minio1`.

   ```shell
   mc support perf net minio1
   ```

3. Drive test

   Run drive read/write performance measurements on all drive on all nodes for a cluster with alias `minio1`. The command uses the default blocksize of 4MiB.

   ```shell
   mc support perf drive minio1
   ```

4. Object test

   Measure the performance of S3 read/write of an object on the alias `minio1`. MinIO autotunes concurrency to obtain maximum throughput and IOPS (Input/Output Per Second).

   ```shell
   mc support perf object minio1
   ```