2017-12-18 23:04 CET

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0001403Orfeo Toolbox (OTB)Generalpublic2017-07-10 11:13
Reporterpoughov 
Assigned Togpasero 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusclosedResolutionfixed 
Summary0001403: InXML1 tests fail but report as passed
DescriptionTests related to xml inputs arguments for applications are passing on the dashboard, but actually they are failing.

I think that input filenames are hardcoded into the input XML files, and therefore are not valid when changing platforms.

I noticed this when working on RFC-91, which changes the error system to an exception which broke (even more) the test and showed it as fail.
Steps To Reproduce$ ctest -R InXML1
=> all tests pass

Manually run the commands found in Modules/Applications/AppClassification/test/CMakeLists.txt (line 206)
=> command fails

(see below for example)
Additional Informationpoughov@to-2216251 ~/build/SuperBuild-develop/OTB/build ctest -R InXML1
Test project /home/poughov/build/SuperBuild-develop/OTB/build
    Start 57: apTvClTrainMethodBOOSTImagesClassifierQB1_InXML1
1/9 Test 0000057: apTvClTrainMethodBOOSTImagesClassifierQB1_InXML1 ..... Passed 1.07 sec
    Start 61: apTvClTrainMethodDTImagesClassifierQB1_InXML1
2/9 Test 0000061: apTvClTrainMethodDTImagesClassifierQB1_InXML1 ........ Passed 0.27 sec
    Start 65: apTvClTrainMethodANNImagesClassifierQB1_InXML1
3/9 Test 0000065: apTvClTrainMethodANNImagesClassifierQB1_InXML1 ....... Passed 0.28 sec
    Start 69: apTvClTrainMethodBAYESImagesClassifierQB1_InXML1
4/9 Test 0000069: apTvClTrainMethodBAYESImagesClassifierQB1_InXML1 ..... Passed 0.28 sec
    Start 73: apTvClTrainMethodRFImagesClassifierQB1_InXML1
5/9 Test 0000073: apTvClTrainMethodRFImagesClassifierQB1_InXML1 ........ Passed 0.27 sec
    Start 77: apTvClTrainMethodKNNImagesClassifierQB1_InXML1
6/9 Test 0000077: apTvClTrainMethodKNNImagesClassifierQB1_InXML1 ....... Passed 0.26 sec
    Start 81: apTvClTrainMethodGBTImagesClassifierQB1_InXML1
7/9 Test 0000081: apTvClTrainMethodGBTImagesClassifierQB1_InXML1 ....... Passed 3.35 sec
    Start 85: apTvClTrainMethodSHARKRFImagesClassifierQB1_InXML1
8/9 Test 0000085: apTvClTrainMethodSHARKRFImagesClassifierQB1_InXML1 ... Passed 0.26 sec
    Start 89: apTvClTrainMethodSHARKKMImagesClassifierQB1_InXML1
9/9 Test 0000089: apTvClTrainMethodSHARKKMImagesClassifierQB1_InXML1 ... Passed 0.26 sec

100% tests passed, 0 tests failed out of 9

Label Time Summary:
OTBAppClassification = 6.30 sec (9 tests)

Total Test time (real) = 6.63 sec

poughov@to-2216251 ~/build/SuperBuild-develop/OTB/build ./bin/otbcli_TrainImagesClassifier -inxml ~/cnes/dev/otb-data/Input/clANN_OutXML1.xml
Can not read file /home/rashad/repos/orfeo/OTB-Data/Input/Classification/VectorData_QB1.shp with GDALOpen
Can not read file /home/rashad/repos/orfeo/OTB-Data/Input/Classification/VectorData_QB1.shp with GDALOpen
ERROR: Directory doesn't exist : /home/rashad/repos/orfeo/build/OTB_/test-build/Testing/Temporary
ERROR: Troubles loading parameter, please check your line argument...
[... further output ommited ...]
TagsNo tags attached.
Attached Files

-Relationships
related to 0001418closedgpasero Two more InXML incorrect tests 
+Relationships

-Notes

~0004175

gpasero (administrator)

This test is weird.

I think the errors come from the wrong paths that are stored in OTB-Data/Input/Classification/cl*_OutXML1.xml .
However, after the parameter -inxml , we set all the input files io.il, io.vd, io.imstats, so this is why the test passes.

Maybe the test strategy should be modified. Before it was using the actual "cl*_OutXML1.xml" produced by the previous test.

~0004176

gpasero (administrator)

Proposition :
* Remove the tests apTvClTrainMethod[...]QB1_OutXML1 : it tests exactly the same thing as apTvClTrainMethod[...]QB1
* Modify the tests apTvClTrainMethod[...]QB1 to use the -outxml param.
* Add the dependency between apTvClTrainMethod[...]QB1_InXML1 and apTvClTrainMethod[...]QB1
* In apTvClTrainMethod[...]QB1_InXML1 : use the xml file produced by apTvClTrainMethod[...]QB1, in Testing/Temporary, instead of the ones in OTB-Data

~0004177

poughov (administrator)

+1 for this solution

~0004182

gpasero (administrator)

Modification of test strategy is done.

However, I had to do a fix regarding the mechanism of InputXMLParameters : in order to support dependant parameters, the Application->UpdateParameters() function has to be called after each value from the XML is set in a parameter.

This fix is working fine on CLI, but not on GUI, appearently due to the fix of 0001025. I'll then modify otb::Wrapper::QtWidgetListViewParameter .

~0004184

gpasero (administrator)

Fix pushed in release-6.0 and develop.

Aside from the test strategy, several fixes have been made regarding handling of InXML parameters and their interactions with ListView parameters :
https://git.orfeo-toolbox.org/otb.git/commit/36935256e237539e37ba15cee6fec913deff4954
https://git.orfeo-toolbox.org/otb.git/commit/489d2705488e3f16f09ad86de72dff6230f10168
https://git.orfeo-toolbox.org/otb.git/commit/39e12c889e181897dbce1cbf13c7c9bcc64b80a5
+Notes

-Issue History
Date Modified Username Field Change
2017-05-18 15:46 poughov New Issue
2017-05-18 15:47 poughov Summary InXML1 fail but report as passed => InXML1 tests fail but report as passed
2017-05-23 11:55 gpasero Note Added: 0004175
2017-05-23 14:48 gpasero Note Added: 0004176
2017-05-23 15:13 poughov Note Added: 0004177
2017-05-24 11:43 gpasero Note Added: 0004182
2017-05-24 11:43 gpasero Assigned To => gpasero
2017-05-24 11:43 gpasero Status new => assigned
2017-05-24 16:43 gpasero Note Added: 0004184
2017-05-24 16:43 gpasero Status assigned => resolved
2017-05-24 16:43 gpasero Resolution open => fixed
2017-05-30 16:48 poughov Relationship added related to 0001418
2017-07-10 11:13 grizonnetm Status resolved => closed
+Issue History