Question: trying to make a higher-order function in a Java module

Friday, 15 November 2024 · 5 messages · last reply 20 Nov 2024
NR
Nico Verwer (Rakensi)15 Nov 2024 14:36 #
This is a long question, and quite complicated, at least to me. I am trying to make a Java module, which is included in BaseX as a jar-file in lib/custom. The module contains one XQuery function:
    named-entity-recognition($grammar as item(), $options as map(*)?) as
    function(item()) as node()*
This is a higher-order XQuery function that returns another (anonymous) XQuery function. The returned function is a parser (for named entities), similar to what the invisible-xml function returns. The relevant part (I think) is the following Java function: @Requires(Permission.NONE) @Deterministic @ContextDependent publicFuncItem namedEntityRecognition(Object grammar, Map<String, String> options) throwsQueryException { // Types of the arguments of the generated function.
    finalVar[] generatedFunctionParameters= {
    newVarScope().addNew(newQNm("input"), SeqType.ITEM_O, queryContext,
    null) };
// Type of the generated function. finalFuncType generatedFunctionType= FuncType.get(SeqType.NODE_ZM, generatedFunctionParameters[0].declType); // The generated function.
    NamedEntityRecognitionFunction nerf=
    newNamedEntityRecognitionFunction(grammar, options,
    generatedFunctionType, queryContext);
// Return a function item.
    returnnewFuncItem(null, nerf, generatedFunctionParameters,
    AnnList.EMPTY, generatedFunctionType,
    generatedFunctionParameters.length, null);
} The NamedEntityRecognitionFunction has a public Value value(final QueryContext qc) function that does the actual parsing. It starts by getting the value of its (only) parameter: Value inputValue = arg(0).value(qc); // Java The namedEntityRecognition function works fine when I use it like this (XQuery):
    let $ner-parse := ner:named-entity-recognition($grammar, map{})
    return $input => $ner-parse()
However, is does not work when the parsing function is put into a declared variable, like
    declare variable $ner-parse as function(item()) as node()* :=
    ner:named-entity-recognition($grammar,map{});
    $input => $ner-parse()
In this case, the parameter value obtained by arg(0).value(qc) is null. When I debug this expression, it turns out that the parameter ($input) is not on the stack. The stack (which is qc.stack) is all nulls, and the begin and end indexes of the stack are both 0, meaning that the stack is empty.. This probably means that the query context of a local (let) variable is different from the query context of a global (declare) variable. Maybe I need to indicate that the namedEntityRecognition function returns a higher order function (but how)? The code for this can be found at https://github.com/nverwer/basex-ner-xar. If anyone can shed some light on this, that would be very much appreciated. Best regards, Nico Verwer
NR
Nico Verwer (Rakensi)15 Nov 2024 15:58 #
I forgot to mention that I am using BaseX 11.5, and compile my code against the 11.5 tag in the github repo of BaseX..
GR
Gunther Rademacherin reply to Nico Verwer (Rakensi), 15 Nov 2024, 14:3615 Nov 2024 19:17 #
Hi Nico, the code in your project seems to have dependencies that cannot be resolved. This is from the log of mvn install: [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4.539 s [INFO] Finished at: 2024-11-15T18:43:46+01:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project basex-ner-xar: Could not resolve dependencies for project com.rakensi:basex-ner-xar:jar:1.0.2 [ERROR] dependency: org.greenmercury:SMAX:jar:1.0.4 (compile) [ERROR] Could not find artifact org.greenmercury:SMAX:jar:1.0.4 in central (https://repo.maven.apache.org/maven2) [ERROR] dependency: com.rakensi:XML-NER:jar:1.0.5 (compile) [ERROR] Could not find artifact com.rakensi:XML-NER:jar:1.0.5 in central (https://repo.maven.apache.org/maven2) [ERROR] [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException Anyway, would it maybe possible for you to provide an even smaller, self-contained example that demonstrates the failure? I have tried the following, which works both with a dynamic and a static variable holding the function item: package test; import static org.basex.query.value.type.SeqType.*; import org.basex.query.*; import org.basex.query.expr.*; import org.basex.query.util.list.*; import org.basex.query.value.*; import org.basex.query.value.item.*; import org.basex.query.value.type.*; import org.basex.query.var.*; import org.basex.util.hash.*;
public class Test extends QueryModule

{

 public FuncItem makeFunction() throws QueryException {
final Var var = new VarScope().addNew(new QNm("input"), STRING_O, queryContext, null); final Var[] params = { var }; final Expr[] exprs = new Expr[] { new VarRef(null, var) }; final FuncType funcType = FuncType.get(SeqType.STRING_O, STRING_O); final TestFunction tf = new TestFunction(exprs); return new FuncItem(null, tf, params, AnnList.EMPTY, funcType, params.length, null); }
 private static final class TestFunction extends Arr {

 protected TestFunction(final Expr[] args) {
super(null, SeqType.STRING_O, args); } @Override
 public Value value(final QueryContext qc) throws QueryException {
return arg(0).value(qc); } @Override
 public Expr copy(final CompileContext cc, final IntObjMap<Var> vm) {
return copyType(new TestFunction(copyAll(cc, vm, args()))); } @Override
 public void toString(final QueryString qs) {
qs.token("test-function").params(exprs); } } } Best regards Gunther Gesendet: Freitag, 15. November 2024 um 14:36
Quoted text · Nico Verwer (Rakensi) · 49 lines
Von: "Nico Verwer (Rakensi)" <nverwer at rakensi.com> An: basex-talk at mailman.uni-konstanz.de Betreff: [basex-talk] Question: trying to make a higher-order function in a Java module This is a long question, and quite complicated, at least to me. I am trying to make a Java module, which is included in BaseX as a jar-file in `lib/custom`. The module contains one XQuery function: > named-entity-recognition($grammar as item(), $options as map(*)?) as function(item()) as node()* This is a higher-order XQuery function that returns another (anonymous) XQuery function. The returned function is a parser (for named entities), similar to what the `invisible-xml` function returns. The relevant part (I think) is the following Java function: > @Requires(Permission.NONE) > > @Deterministic > > @ContextDependent > > public FuncItem namedEntityRecognition(Object grammar, Map<String, String> options) throws QueryException { > > // Types of the arguments of the generated function. > > final Var[] generatedFunctionParameters = { new VarScope().addNew(new QNm("input"), SeqType.ITEM_O, queryContext, null) }; > > // Type of the generated function. > > final FuncType generatedFunctionType = FuncType.get(SeqType.NODE_ZM, generatedFunctionParameters[0].declType); > > // The generated function. > > NamedEntityRecognitionFunction nerf = new NamedEntityRecognitionFunction(grammar, options, generatedFunctionType, queryContext); > > // Return a function item. > > return new FuncItem(null, nerf, generatedFunctionParameters, AnnList.EMPTY, generatedFunctionType, generatedFunctionParameters.length, null); > > } The NamedEntityRecognitionFunction has a public Value value(final QueryContext qc) function that does the actual parsing. It starts by getting the value of its (only) parameter: > Value inputValue = arg(0).value(qc); // Java The namedEntityRecognition function works fine when I use it like this (XQuery): > let $ner-parse := ner:named-entity-recognition($grammar, map{}) > return $input => $ner-parse() However, is does not work when the parsing function is put into a declared variable, like > declare variable $ner-parse as function(item()) as node()* := ner:named-entity-recognition($grammar,map{}); > $input => $ner-parse() In this case, the parameter value obtained by arg(0).value(qc) is null. When I debug this expression, it turns out that the parameter ($input) is not on the stack. The stack (which is qc.stack) is all nulls, and the begin and end indexes of the stack are both 0, meaning that the stack is empty.. This probably means that the query context of a local (let) variable is different from the query context of a global (declare) variable. Maybe I need to indicate that the namedEntityRecognition function returns a higher order function (but how)? The code for this can be found at https://github.com/nverwer/basex-ner-xar. If anyone can shed some light on this, that would be very much appreciated. Best regards, Nico Verwer
NR
Nico Verwer (Rakensi)18 Nov 2024 10:27 #
Hello Gunther, Thank you for your response. It gives me something to look at. I am currently working on 3 different projects, so I will probably continue with this at the end of the week.
Quoted text · Gunther Rademacher · 2 lines
> the code in your project seems to have dependencies that cannot be > resolved.
That is indeed a problem. I should make these dependencies available somehow. Maybe Github or Maven central. Thanks for your example code. I will see if I can use it to reproduce (and fix?) my problem. Best regards, Nico
NR
Nico Verwer (Rakensi)in reply to Gunther Rademacher, 15 Nov 2024, 19:1720 Nov 2024 15:30 #
Dear Gunther, Thank you for the test code. I gradually changed it into my code that did not work, and found the problem:
         private static Expr[] parameterVars(FuncType funcType,
    QueryContext queryContext)
         {
           Expr[] paramVars = new Expr[funcType.argTypes.length];
           for (int i = 0; i < paramVars.length; ++i) {
             paramVars[i] = new VarRef(null, new VarScope().addNew(new
    QNm("arg"+i), funcType.argTypes[i], queryContext, null));
           }
           return paramVars;
         }
I was trying to be clever, and derive the parameter VarRefs from the function type. But this cannot get to the original parameter names, and I invented new names; new QNm("arg"+i). But these names are different from the original names, and somehow this means that in some cases there will be no parameters on the stack. What is strange / interesting, is that with the above code, the following works:
    let $ner-parse := ner:named-entity-recognition($grammar, map{})
    return $input => $ner-parse()
But using a declared variable does not work:
    declare variable $ner-parse as function(item()) as node()* :=
    ner:named-entity-recognition($grammar,map{});
    $input => $ner-parse()
Now I make sure I use the same names for the generated VarRefs, and my function is always called with the parameter values on the stack. Thanks for your help! Best regards, Nico

⚡Generated with XQuery