Tuesday, August 6, 2013

11. Regression Tests

Regression tests are a set of tests that exercise the functionality of the released gateware and software. The following are the initial tests that need to be followed :
1.Go to : Systems -> Administration -> Server Settings -> Services  and stop all the processes (except the process network) that are running (time consuming but has to be done). I had once faced a problem of not being able to open the Systems  due to insufficient rights (dunno how that happened as I was the only user on the computer). For that, you can log into the  root and type : system-config-services.
2. IPv6 needs to be disabled. This can be done as follows :
a. Type : /sbin/chkconfig ip6tables off
b. Type : gedit /etc/modprobe.conf and add the lines :
alias ipv6 off
alias net-pf-10 off


 c. Edit network file using the command : gedit /etc/sysconfig/network and make the sure the following line is there : NETWORKING_IPV6=no



Regression Tests
1. Connect 'eth1' to 'nf2c0' and 'eth2' to 'nf2c1'.
2. Connect 'nf2c2' to 'nf2c3' (Leaving these ports unconnected causes errors in the regression tests).
3. Pull up all the interfaces :
ifconfig eth1 up
ifconfig eth2 up
ifconfig nf2c0 up
ifconfig nf2c1 up
ifconfig nf2c2 up
ifconfig nf2c3 up
And assign an IP address of 0.0.0.0 to all of them. This is done to remove the IPv4 address for each interface.
ifconfig eth1 0.0.0.0
ifconfig eth2 0.0.0.0
ifconfig nf2c0 0.0.0.0
ifconfig nf2c1 0.0.0.0
ifconfig nf2c2 0.0.0.0
ifconfig nf2c3 0.0.0.0
Now reprogram the CPCI using : /usr/local/sbin/cpci_reprogram.pl -all
Write the bitfile onto the NetFPGA using : nf_download ~/netfpga/bitfiles/reference_router.bit
Now start the regression tests using : ~/netfpga/bin/nf_regress_test.pl 
RESULTS

41 comments:

  1. hey do you know how to run this test with the release 3.0?

    ReplyDelete
  2. Hi,
    I have this error coming
    can’t locate NF/Base.pm in @INC (@INC contains: /usr/lib/perl5……………….) at /root/netfpga/bin/nf_regress_test.pl line 3.
    BEGIN failed — compilation aborted at /root/netfpga/bin/nf_regress_test.pl line 3.

    ReplyDelete
    Replies
    1. i rmmbr i had faced this problem but i don’t rmmbr how i got arnd it.. did u disable all the services under Administration?

      Delete
  3. im facing the problem that while running the command ~/netfpga/bin/nf_regress_test.pl –project reference_router.
    I get an error:

    nf_regress_test.pl: Cannot locate regression test file ‘/root/netfpga/projects/reference_router/regress/tests.txt’ for project ‘reference_router’

    I searched the internet but they say that this file is not available in the v.3.0 package. Now tell me what should i do?

    ReplyDelete
    Replies
    1. i dont think the version of the package makes any difference because i ran the same tests for v3.0 and didn’t have any problems.. just make sure that you stop all the services under Administration.. that was one thing that created a lot of problems for me..

      Delete
  4. Hi Hari.

    I have only eth0, because I have only one network card with one port RJ-45.

    What I can do? Can I connect ‘eth0′ to ‘nf2c0′ and nf2c1 to another PC ?

    ReplyDelete
    Replies
    1. u will need 2 ports to complete the regression tests.. but then regression tests are useful just to confirm whether the application is working fine or not.. u can skip them but the problem would b that you won’t know what the errors are.. i suggest you use a network card with 2 ethernet ports.. i dunno if u can connect nf2c1 to another pc because i have never tried it.. you can try it.. if it works out then well and good..

      Delete
    2. Hi pharic, I got another network card (NIC), now I have eth0 and eth1, but when i did regression tests output this:

      [root@localhost /]# ~/netfpga/bin/nf_regress_test.pl

      NetFPGA environment:
      Root dir: /root/netfpga
      Project name: reference_nic
      Project dir: /root/netfpga/projects/reference_nic
      Work dir: /tmp/root

      Root directory is /root/netfpga

      nf_regress_test.pl: Cannot locate regression test file ‘/root/netfpga/projects/router_buffer_sizing/regress/tests.txt’ for project ‘router_buffer_sizing’
      [root@localhost /]#

      Do you know this error?

      Delete
    3. hey.. i rmmbr getting a similar problem but dont rmmbr how i got around it.. try googling the error.. might be helpful..

      Delete
    4. I have a same problem, and googling won't help because netfga forum is off. You still don't know how to fix this?

      Delete
    5. i m sry XXXXXX but what problem do u have? one eth port or cannot locate regression test file?

      Delete
    6. cannot locate regression test file:(

      Delete
    7. first chk manually if the file is actually located in netfpga/bin directory.. if it is then there's a possibility tht there's some prob with the installation.. also make sure all processes under services are closed..

      Delete
    8. There is not such file... I have progblem during installation of netfpga base package 2.1.1 (I already told you about netfpga kernel problem) so I removed that packet and installed new using yum install netfpga-base (v 3.0.1). Could that be problem? Don't know what to do know... To reinstall OS and start from beginning? And if I do that what packet to install 3.0.1 or 2.1.1. I want to use netfpga as reference router...

      Delete
    9. run : yum install netfpga-kernel - and tell me wht output u get..

      Delete
    10. I am home now so netfpga board isn't with me. I will tell u that tomorrow. Can u aswer me a last question about choosing 3.0.1 or 2.1.1 version for reference router until then? Could I use 3.0.1? Maybe than I won't have problem with netfpga kernel installation?

      Delete
    11. its always better to go with the recommended version and fr reference router it is 2.1.1.. don't go with reinstallation until u r sure thts the only thing u can do.. tell me the output with tht command tomorrow and let's see if we can get it running..

      Delete
    12. Hi :) I removed netfpga base package today, and installed netfpga-base-2.1.1 and netfpga-kernel-2.1.1 and solved problem with locating regression test. Now I have new one ... Some tests failed. At first it reported problem that I don't have eth2 interface (I had eth0 and eth1). I deleted ifcfg-eth0 file and copied its contents in ifcfg-eth1, also copied contents of eth1 (before) in new file ifcfg-eth2 that I created manually. I don't know is this causing problems? Also I failed to assign 0.0.0.0 Ip adresses to interfaces, I typed comands from your blog but ifcfg command shows me that all interfaces have other adresses and get it by dhcp. Is this problem? Driver tests and some scone test gone well and than some problem with dijkstra routing problem, also I got messages about missing ARP request/reply pairs, after that more failed tests.

      Delete
    13. renaming et0 and eth1 as eth1 and eth2 doesn't solve the problems.. never mess with them.. :D the regression test behaviour can be very erratic.. was the board programmed successfully with the bitfile?

      Delete
    14. This comment has been removed by the author.

      Delete
  5. Hi pharic, I have a problem with nf_download ~/netfpga/bitfiles/reference_router.bit
    This commands show the follow error ;

    Bit file built from: nf2_top_par.ncd;HW_TIMEOUT=FALSE.

    Maybe, fails the Regression test.

    Can you help me ?

    [root@localhost /]# nf_download ~/netfpga/bitfiles/reference_router.bit
    Found net device: nf2c0
    Bit file built from: nf2_top_par.ncd;HW_TIMEOUT=FALSE
    Part: 2vp50ff1152
    Date: 2011/11/17
    Time: 17:49:43
    Error Registers: 0
    Good, after resetting programming interface the FIFO is empty
    Download completed – 2377668 bytes. (expected 2377668).
    DONE went high – chip has been successfully programmed.
    CPCI Information
    —————-
    Version: 4 (rev 1)

    Device (Virtex) Information
    —————————
    Project directory: reference_router
    Project name: Reference router
    Project description: Reference IPv4 router

    Device ID: 2
    Version: 1.0.0
    Built against CPCI version: 4 (rev 1)

    Virtex design compiled against active CPCI version

    ReplyDelete
    Replies
    1. try – nf2_download ~/netfpga/bitfiles/reference_router.bit

      dunno if it will work.. worth a try..

      Delete
  6. hi pharic,
    i have this problem:

    [root@localhost synth]# ~/netfpga/bin/nf_regress_test.pl

    NetFPGA environment:
    Root dir: /root/netfpga
    Project name: reference_nic
    Project dir: /root/netfpga/projects/reference_nic
    Work dir: /tmp/root

    Root directory is /root/netfpga

    nf_regress_test.pl: Cannot locate regression test file ‘/root/netfpga/projects/router_buffer_sizing/regress/tests.txt’ for project ‘router_buffer_sizing’

    can help me pleace, tnks

    ReplyDelete
    Replies
    1. cud b some process runnibg under Services.. i m not sure though...

      Delete
  7. Hi pharic i noticed that release 3.0 dont use the perl script to test anymore..

    Acording to the developers they using nf_test.py script to run the tests…

    But when i run the python script im getting this error:

    [root@localhost bin]# python nf_test.py hw

    NetFPGA environment:
    Root dir: /root/netfpga
    Project name: reference_nic
    Project dir: /tmp/raphael/test/reference_nic

    Work dir: /tmp/raphael
    Root directory is /root/netfpga
    Running global setup… /tmp/raphael/test/reference_nic/global/setup exited with value 1
    FAIL
    Output was:

    NetFPGA environment:
    Root dir: /root/netfpga
    Project name: reference_nic
    Project dir: /root/netfpga/projects/reference_nic
    Work dir: /tmp/raphael

    Found net device: nf2c0
    Virtex design compiled against active CPCI version
    Bit file built from: nf2_top_par.ncd;HW_TIMEOUT=FALSE
    Part: 2vp50ff1152
    Date: 2011/11/17
    Time: 16:21:17
    Error Registers: 0
    Good, after resetting programming interface the FIFO is empty
    Strange. FIFO wasnt empty… trying again.
    Strange. FIFO wasnt empty… trying again.
    Strange. FIFO wasnt empty… trying again.
    Strange. FIFO wasnt empty… trying again.
    Strange. FIFO wasnt empty… trying again.
    Strange. FIFO wasnt empty… trying again.
    Strange. FIFO wasnt empty… trying again.
    Download completed – 2377668 bytes. (expected 2377668).
    DONE went high – chip has been successfully programmed.
    CPCI Information
    —————-
    Version: 4 (rev 1)

    Device (Virtex) Information
    —————————
    Project directory: reference_nic
    Project name: Reference NIC
    Project description: Reference NIC

    Device ID: 1
    Version: 1.1.0
    Built against CPCI version: 4 (rev 1)

    SIOCSIFADDR: Dispositivo inexistente
    eth2: ERROR while getting interface flags: Dispositivo inexistente
    SIOCSIFNETMASK: Dispositivo inexistente
    eth2: erro obtendo informações da interface: dispositivo não encontrado
    SIOCSIFADDR: Dispositivo inexistente
    eth1: ERROR while getting interface flags: Dispositivo inexistente
    SIOCSIFNETMASK: Dispositivo inexistente
    eth1: erro obtendo informações da interface: dispositivo não encontrado
    FAIL

    I have a net card with only eth0 interface, dunno if this an issue.

    ReplyDelete
    Replies
    1. if u have only 1 ethernet port then u cant run these tests.. u need to insert an ethernet card with a dual port..

      Delete
  8. [root@it46 ~]# ~/netfpga/bin/nf_regress_test.pl
    Can’t locate NF/Base.pm in @INC (@INC contains: /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.8/i386-linux-thread-multi /usr/lib/perl5/5.8.8 .) at /root/netfpga/bin/nf_regress_test.pl line 3.
    BEGIN failed–compilation aborted at /root/netfpga/bin/nf_regress_test.pl line 3.

    we are getting this error .how to recover from this problem.we have only one port in our pc.we completed self test successfully.we have connected nfc0 to eth0..to connect nfc1 should we require another pc with a same os or different os is possible?

    ReplyDelete
    Replies
    1. u cannot use another pc.. u shud insert an additional ethernet card on ur pc..

      Delete
  9. hi again ,ALL THE OTHER PROJECTS in regression tests are running FINE , even ALL THE other tests of scone have “PASSED” , except THESE TWO , PLEASE ADVISE
    —————————————————————————————————-
    NetFPGA environment:
    Root dir: /root/netfpga
    Project name: scone
    Project dir: /root/netfpga/projects/scone
    Work dir: /tmp/nl29

    Root directory is /root/netfpga
    Running tests on project ‘scone’…

    Running test ‘test_ip_set’… FAIL
    Output was:
    Found net device: nf2c0

    ** sr_integ_init(..) called

    - Name [eth0] IP [192.168.0.2] Mask [255.255.255.0] MAC [00:00:00:00:00:01]
    ** sr_integ_add_interface(..) called
    - Name [eth1] IP [192.168.1.2] Mask [255.255.255.0] MAC [00:00:00:00:00:02]
    ** sr_integ_add_interface(..) called
    - Name [eth2] IP [192.168.2.2] Mask [255.255.255.0] MAC [00:00:00:00:00:03]
    ** sr_integ_add_interface(..) called
    - Name [eth3] IP [192.168.3.2] Mask [255.255.255.0] MAC [00:00:00:00:00:04]
    ** sr_integ_add_interface(..) called

    ** sr_integ_hw(..) called
    —RTABLE AFTER DIJKSTRA—
    Destination Gateway Mask Iface Static Active
    192.168.3.0 0.0.0.0 255.255.255.0 eth3 N Y
    192.168.2.0 0.0.0.0 255.255.255.0 eth2 N Y
    192.168.1.0 0.0.0.0 255.255.255.0 eth1 N Y
    192.168.0.0 0.0.0.0 255.255.255.0 eth0 N Y

    Failed to set the following ip filters: 0xC0A80002L 0xC0A80102L 0xC0A80202L 0xC0A80302L 0xE0000005L

    test_ip_set/run exited with value 1

    Running test ‘test_rtable_set’… FAIL
    Output was:
    Found net device: nf2c0

    ** sr_integ_init(..) called

    - Name [eth0] IP [192.168.0.2] Mask [255.255.255.0] MAC [00:00:00:00:00:01]
    ** sr_integ_add_interface(..) called
    - Name [eth1] IP [192.168.1.2] Mask [255.255.255.0] MAC [00:00:00:00:00:02]
    ** sr_integ_add_interface(..) called
    - Name [eth2] IP [192.168.2.2] Mask [255.255.255.0] MAC [00:00:00:00:00:03]
    ** sr_integ_add_interface(..) called
    - Name [eth3] IP [192.168.3.2] Mask [255.255.255.0] MAC [00:00:00:00:00:04]
    ** sr_integ_add_interface(..) called

    ** sr_integ_hw(..) called
    Read: 192.168.1.1 192.168.1.1 255.255.255.0 eth1
    Read: 0.0.0.0 192.168.0.1 0.0.0.0 eth0
    —RTABLE AFTER DIJKSTRA—
    Destination Gateway Mask Iface Static Active
    192.168.3.0 0.0.0.0 255.255.255.0 eth3 N Y
    192.168.2.0 0.0.0.0 255.255.255.0 eth2 N Y
    192.168.1.1 192.168.1.1 255.255.255.0 eth1 Y Y
    192.168.1.0 0.0.0.0 255.255.255.0 eth1 N Y
    192.168.0.0 0.0.0.0 255.255.255.0 eth0 N Y
    0.0.0.0 192.168.0.1 0.0.0.0 eth0 Y Y

    Failed to find the following routes:
    0xC0A80101L 0xC0A80101L 0xFFFFFF00L 0x4L
    0x0L 0xC0A80001L 0x0L 0x1L

    test_rtable_set/run exited with value 1

    ReplyDelete
    Replies
    1. i dunno abt this.. mayb u need to turn off some process under services..

      Delete
    2. do you knw where i can find the details abt these tests , like for proper debuggin .. , just asking , i am still stuck with these 2 errors : \ , getting really frustrating

      Delete
  10. have you ever tried configuring switches?

    ReplyDelete
    Replies
    1. nope.. my project was for a duration of 2 months n i cudnt do much..

      Delete
  11. Hi;)
    When I load reference_router bitfile should I use ~/netfpga/bin/nf_regress_test.pl command or ~/netfpga/bin/nf_regress_test.pl --project reference_router. I ask u this because when I use first command every test is passed except one for reference_switch.... Now I am not sure am I doing all well. Should I load another bitfile to test reference_switch?

    ReplyDelete
    Replies
    1. hey.. when u run ~/netfpga/bin/nf_regress_test.pl --project reference_router the regression tests specific to the reference router are run.. so u shud b fyn..

      Delete
    2. Yes but what if I want to run NETFPGA like reference switch? After loading reference_switch bitfile regression test for it fails... That's only regression test that failed... It's weird. How could I know is this bitfile ok? I installed Netfga on two PCs, and this test fails on both of them. There are some errors with registers...

      Delete
    3. what exactly are you trying to implement? reference router or switch?

      if u downloaded the bitfile from the project then it will be defintely ok because it would obviously be tested before they put it up.. i dunno y it be failing.. maybe some process under services might be interfering..

      Delete
    4. I am trying to implement reference switch. I've turned off all services except networking, but regression test still fails. It can't find registers that regression test is looking for. Realy don't know what is a problem. I have used netfpga base packet version 2.1.1. Maybe I should use some older version?

      Delete
    5. you could try a different version but its best to use the recommended version.. just try running after disabling networking service too..

      Delete