Azure function java worker Jackson Core 2.9.6 Dependecy

[albertodema]

Hi I am experiencing some blockers while trying to use (for model scoring) spark on Azure Functions.
I managed to find an exception that is basically blocking any spark operation:
com.fasterxml.jackson.databind.JsonMappingException: Incompatible Jackson version: 2.9.6
Azure Functions Java Worker has Jackson 2.9.6 as dependency and spark 2.2 cannot work with anything >2.6.
So even if I try to exclude Jackson Core 2.9 from dependencies in my function pom.xml it is in reality anyway used by the worker and this causes spark to fail.
Is there a way to have the Azure Worker to use Jackson Core 2.6?

[pragnagopa]

Notes from @selvasingh
We should not downgrade this. A luxury version is to make sure developers can plug in a lower version of Jackson, if they like to.

[albertodema]

Hi I was able to have spark working locally with those modifications to the pom.xml file of azure-functions-worker project:

io.grpc
grpc-netty-shaded
1.15.0


com.fasterxml.jackson.core
jackson-databind
2.6.5


com.fasterxml.jackson.core
jackson-annotations
2.6.5

The first one is just the same dependecy but shaded in order to avoid collision with spark that is using netty 4.0, the remaining two are referring to an older version of jackson that works with spark.

Unfortunately I cannot deploy this version on azure functions because the azure-java-worker.jar is placed in a location with readonly permissions.

I am currently stuck at this point and evaluating other cloud alternatives to progress.

[pragnagopa]

cc: @paulbatum
azure-functions-java-worker is part of the functions runtime and cannot be updated by the user.
Unfortunately we do not have a solution. We do plan on addressing issue #183 to decouple function code dependencies from functions runtime dependencies.

[pragnagopa]

@brunoborges / @JonathanGiles - Any ideas on how to package dependencies for azure-fucntions-java-worker in a way that gives the ability for users to override worker dependencies with their own?

[brunoborges]

Only way I can think of is if you ensure user-defined dependencies are loaded first in the clasapath. But that may break the worker if user chooses an incompatible version of a dependency.

In other words: I don't think it is possible.

As I mentioned couple months ago, right now the architecture has worker and user code tightly coupled.

The ideal solution is to have user code running in one Java process, and the worker in another one.

This is how OpenFaaS and FN Project are designed.

[asavaritayal]

Closing issue in favor of #183