Unit module sometimes drives me nuts
Wednesday, 30 July 2014 · 5 messages · last reply 7 Aug 2014
MG
Marc van Grootel30 Jul 2014 14:28 #
Really, I want to be good and write my tests together with the code. I still don't do it often enough but I know it's important and for the code I'm currently working on I try to cover most of it with tests.
I regret that I'm not able to get down into the guts of the Java code and help fix it, so, instead I moan ;-)
No, seriously, here are a few more general things that bug me about the Unit module.
- assertions causing an NPE when an XQuery error is raised but no traceback that tells me what/where the error actually is. [ex1]
- assertions causing a failure/error reported, produce a test result but stop running further tests after the one that raises the error [ex2]
I tried to reduce these issues into a short example but I think that they highlight a more general issue with the current Unit module.
It's mainly the fact that there's no indication of the problem that forces me to add unit:ignore to all tests and one by one removing it, or executing the assertion xquery in a client to find out where the issue lies.
I also have an improvement idea:
- Personally, I run the tests often (minutes rather than hours), and from the shell. The XML format feedback isn't very helpful. It would be for automated test reports but I think the usability (readability) of the feedback during development is just as, if not more, important. Maybe adding an extra switch (-t for textual feedback and -tx for XML feedback) could help. It should just focus on summarizing clearly the expected, return values, failures and errors, and provide helpful context.
Note that I always run the code using
basex -t from bash and I'm running 8.0 beta 7ba9510
Note too that most of these things happen to me when doing map related stuff.
... and now I stop moaning and get back to coding.
--Marc
[ex1] example of test that causes an NPE without any helpful output
*declare* *%unit:test* *function* *test:npe*() {
*unit:assert-equals*(*map* {}('x'), *map* { 'x': 1 }('x')) };
[ex2] example of test that causes further tests to be executed*declare* *%unit:test* *function* *test:fail*() {
*unit:assert-equals*(*map* {}, *map* { 'x': 1 })
};doing
map {} = map {'x': 1} causes this error
[XPTY0004] map(*) and map(*) cannot be compared.
CG
Christian Grün30 Jul 2014 16:48 #
Hi Marc,
Quoted text · Marc van Grootel · 2 lines
> I regret that I'm not able to get down into the guts of the Java code and > help fix it, so, instead I moan ;-)
Thanks for moaning! XQUnit is still sth. we are heavily working on, so any feedback is welcome.
Quoted text · Marc van Grootel · 2 lines
> - assertions causing an NPE when an XQuery error is raised but no traceback > that tells me what/where the error actually is. [ex1]
NPEs are usually serious bugs. We'll have a look at that.
Quoted text · Marc van Grootel · 2 lines
> - assertions causing a failure/error reported, produce a test result but > stop running further tests after the one that raises the error [ex2]
The reason is that maps are functions, and functions cannot be compared. But the current version of the XQuery spec. will provide support for comparing maps, so it's basically a matter of time when the comparision of maps will be made possible. I'll dig into this one as well; maybe it's not a big deal to provide an early yet stable implementation.
Quoted text · Marc van Grootel · 2 lines
> - Personally, I run the tests often (minutes rather than hours), and from > the shell. The XML format feedback isn't very helpful.
It's probably not that trivial to find a proper textual representation for everything that can be done with XQUnit. One thing you can do is to send the output to a second BaseX call. The following command-line code will only return the failures of your XML code:
basex -t . | basex -i - //failure
Of course, you could also call a function that does some more sophisticated stuff with your input.
Hope this helps,
Christian
MG
Marc van Grootel30 Jul 2014 17:06 #
Hi Christian,
Yes, I already learned to write tests to work around not being able to compare maps. It was more the general behaviour of a test run not giving me much to go on. Noticed it on maps mainly.
Thanks for the tip of piping result through an expression, didn't think of that.
Keep up the good work, I'm generally enjoying this :)
--Marc
Quoted text · Christian Grün · 42 lines
On Wed, Jul 30, 2014 at 4:48 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Marc, > > > I regret that I'm not able to get down into the guts of the Java code and > > help fix it, so, instead I moan ;-) > > Thanks for moaning! XQUnit is still sth. we are heavily working on, so > any feedback is welcome. > > > - assertions causing an NPE when an XQuery error is raised but no > traceback > > that tells me what/where the error actually is. [ex1] > > NPEs are usually serious bugs. We'll have a look at that. > > > - assertions causing a failure/error reported, produce a test result but > > stop running further tests after the one that raises the error [ex2] > > The reason is that maps are functions, and functions cannot be > compared. But the current version of the XQuery spec. will provide > support for comparing maps, so it's basically a matter of time when > the comparision of maps will be made possible. I'll dig into this one > as well; maybe it's not a big deal to provide an early yet stable > implementation. > > > - Personally, I run the tests often (minutes rather than hours), and from > > the shell. The XML format feedback isn't very helpful. > > It's probably not that trivial to find a proper textual representation > for everything that can be done with XQUnit. One thing you can do is > to send the output to a second BaseX call. The following command-line > code will only return the failures of your XML code: > > basex -t . | basex -i - //failure > > Of course, you could also call a function that does some more > sophisticated stuff with your input. > > Hope this helps, > Christian >
--
--Marc
CG
Christian Grün6 Aug 2014 20:08 #
Hi Marc,
XQUnit can now also be used to compare maps. Feel free to check out the latest snapshot [1].
Cheers,
Christian
[1] http://files.basex.org/releases/latest/
Quoted text · Marc van Grootel · 67 lines
On Wed, Jul 30, 2014 at 5:06 PM, Marc van Grootel <marc.van.grootel at gmail.com> wrote: > Hi Christian, > > Yes, I already learned to write tests to work around not being able to > compare maps. It was more the general behaviour of a test run not giving me > much to go on. Noticed it on maps mainly. > > Thanks for the tip of piping result through an expression, didn't think of > that. > > Keep up the good work, I'm generally enjoying this :) > > --Marc > > > > On Wed, Jul 30, 2014 at 4:48 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Hi Marc, >> >> > I regret that I'm not able to get down into the guts of the Java code >> > and >> > help fix it, so, instead I moan ;-) >> >> Thanks for moaning! XQUnit is still sth. we are heavily working on, so >> any feedback is welcome. >> >> > - assertions causing an NPE when an XQuery error is raised but no >> > traceback >> > that tells me what/where the error actually is. [ex1] >> >> NPEs are usually serious bugs. We'll have a look at that. >> >> > - assertions causing a failure/error reported, produce a test result but >> > stop running further tests after the one that raises the error [ex2] >> >> The reason is that maps are functions, and functions cannot be >> compared. But the current version of the XQuery spec. will provide >> support for comparing maps, so it's basically a matter of time when >> the comparision of maps will be made possible. I'll dig into this one >> as well; maybe it's not a big deal to provide an early yet stable >> implementation. >> >> > - Personally, I run the tests often (minutes rather than hours), and >> > from >> > the shell. The XML format feedback isn't very helpful. >> >> It's probably not that trivial to find a proper textual representation >> for everything that can be done with XQUnit. One thing you can do is >> to send the output to a second BaseX call. The following command-line >> code will only return the failures of your XML code: >> >> basex -t . | basex -i - //failure >> >> Of course, you could also call a function that does some more >> sophisticated stuff with your input. >> >> Hope this helps, >> Christian > > > > > -- > --Marc
MG
Marc van Grootel7 Aug 2014 11:32 #
Hi Christian,
Got it. Rewrote some tests and it works great.
Thanks.
--Marc
Quoted text · Christian Grün · 85 lines
On Wed, Aug 6, 2014 at 8:08 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Marc, > > XQUnit can now also be used to compare maps. Feel free to check out > the latest snapshot [1]. > > Cheers, > Christian > > [1] http://files.basex.org/releases/latest/ > > > On Wed, Jul 30, 2014 at 5:06 PM, Marc van Grootel > <marc.van.grootel at gmail.com> wrote: > > Hi Christian, > > > > Yes, I already learned to write tests to work around not being able to > > compare maps. It was more the general behaviour of a test run not giving > me > > much to go on. Noticed it on maps mainly. > > > > Thanks for the tip of piping result through an expression, didn't think > of > > that. > > > > Keep up the good work, I'm generally enjoying this :) > > > > --Marc > > > > > > > > On Wed, Jul 30, 2014 at 4:48 PM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> Hi Marc, > >> > >> > I regret that I'm not able to get down into the guts of the Java code > >> > and > >> > help fix it, so, instead I moan ;-) > >> > >> Thanks for moaning! XQUnit is still sth. we are heavily working on, so > >> any feedback is welcome. > >> > >> > - assertions causing an NPE when an XQuery error is raised but no > >> > traceback > >> > that tells me what/where the error actually is. [ex1] > >> > >> NPEs are usually serious bugs. We'll have a look at that. > >> > >> > - assertions causing a failure/error reported, produce a test result > but > >> > stop running further tests after the one that raises the error [ex2] > >> > >> The reason is that maps are functions, and functions cannot be > >> compared. But the current version of the XQuery spec. will provide > >> support for comparing maps, so it's basically a matter of time when > >> the comparision of maps will be made possible. I'll dig into this one > >> as well; maybe it's not a big deal to provide an early yet stable > >> implementation. > >> > >> > - Personally, I run the tests often (minutes rather than hours), and > >> > from > >> > the shell. The XML format feedback isn't very helpful. > >> > >> It's probably not that trivial to find a proper textual representation > >> for everything that can be done with XQUnit. One thing you can do is > >> to send the output to a second BaseX call. The following command-line > >> code will only return the failures of your XML code: > >> > >> basex -t . | basex -i - //failure > >> > >> Of course, you could also call a function that does some more > >> sophisticated stuff with your input. > >> > >> Hope this helps, > >> Christian > > > > > > > > > > -- > > --Marc >
--
--Marc
Written for BaseX 7.9. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery