3.1 Test Methods and Setup

Posted by JTSF at Saturday, June 22, 2013 Read our previous post

Testing Method is finalized.
Minor revision1: I discovered a bottleneck on my Testing desktop HDD. Changed Testing Desktop HDD from WD 3TB to Toshiba 3TB.
Minor revision2: Revised Test Files to include 7 files that are larger than 4gb.
Minor revision3: Removed Performance Tests 11 & 12 due to NAS Performance Tester does not support iSCSI.
Minor Addition: Calculation of average speed for test 1 & 2.  

Test Methods
Testing is split into two sections, performance tests and functionality tests. Recalled that I have 3 different setups and all three setups will be put through the same set of test cases. 


1. Performance Tests
In total, there are ten performance tests cases. For specific tests details, refer to them after the table. For each setup, I will test both RAID 1 & RAID 5 (or equivalent) performance.

1 & 2:  CIFS Copy To / From NAS

• Objective of test 1 & 2: assess actual usage performance.
• Test files are 100gb in total; Consist of  ISO (larger than 4gb), movies (1-3gb) and a long list of small files (pictures)
• Average speed is recorded after 10mins.
• Power consumption is recorded during test.
• Duration will be recorded.
• Observations will be note down.
• 100GB = 102400 Megabyte
• Average speed is calculated by: 102400 / time taken (in secs) 
• Eg: Time taken is 18mins 10secs;  102400 / (18x60) + 10 =  93.94MB/sec 

3 - 6:  CIFS / CrystalDiskMark, NAS Performance Tester

• Full tests results will be given in printscreens. 
• Run test twice, highest score will be taken.

7-10:  iSCSI

• Similar testing methodology to 1-6 will be applied.

2. Functionality Tests
For functionality tests, my focus is mainly on the resiliency of the built and what features does the OS offers.  
- All functionality test cases are conducted under RAID 5 (or equivalent) setups.
- 100gb of test files are stored on storage pool as a form of verification.

1: Motherboard Failure

• Objective of test: simulate motherboard failure.
• Test procedures: Swap boot drive (USB) and storage drives (HDDs) to another motherboard of different chipset.
• OS and all data must be preserved to pass test.

 2: HDD Failure

• Objective of test: simulate 1x HDD failure.
• Test procedures: i. Remove and format 1x HDD     ii. Add HDD back to storage pool
• All data (100gb) must be preserved to pass test.
• With 1 drive failure, can files still be written to degraded storage pool?
• Record rebuild time.

3: Storage Scalability

• Objective of test: Adding one or more HDDs to storage pool.
• Test procedures: Add 1x 2TB HDD to storage pool.
• Assess complexity of procedures to scale storage pool.

4: Hot-swap HDD

• Objective of test: Hot-swap HDD (on the fly) while NAS is in operational mode.
• Test procedures: i. Plug in 1x 2TB HDD to NAS     ii. Remove 1x 2TB HDD from NAS
• NAS must be able to detect addition / removal of HDD to pass test.

5: NAS Hibernate mode

• Objective of test: Assess if the NAS is able to enter hibernation mode with HDDs spin down.
• Test procedures: Observe NAS (HDD state, power consumption) after 1 hour of inactivity. 
• NAS must be able to hibernate and spin down HDDs to pass test.
• Record power consumption of hibernate mode.

6: NAS Operating System

• Share personal experience of OS

Testbed Setup

  Power Consumption Measurement Tool
  DX Power Consumption Monitor Meter

Testbed Topology

Testing Desktop HDD Validation Test
WD 3TB to Toshiba 3TB, disk to disk file transfer test on Testing Desktop to ensure that it will not be the bottleneck.

Any questions on the testing? Send me a query.