Proteus Springboot - RSocket "can not find service" errors


#1

Hi all,

Been playing around with the Spring Boot quickstart for Proteus. Been running into issues trying to get 2 services talking to each other. I’m building the project using Maven.

When I try to send a message to my 2nd service (which has its own IDL module like in the quickstart - but is pretty much a copy of the HelloService), I get an RSocket error saying it “can not find the service,” despite everything compiling fine and running otherwise. My initial guess is it’s something to do with protobuf and how it handles namespaces but I can’t wrap my head around where the actual issue could be. Might be something really obvious…

For example, I noticed from a clean pull from your Spring Boot quickstart, if I change the java_package of the protobuf file to e.g. “io.netifi.proteus.contract” (then compile and change all the imports in client/service to reflect this), I get the error below. I saw this and thought, the java_package and package fields must have to match the service exactly, but nothing I’ve tried gets my second service to work.

Hopefully this all makes sense.

Error on the client:
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:816)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:797)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:324)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248)
at io.netifi.proteus.quickstart.client.Main.main(Main.java:10)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.rsocket.exceptions.ApplicationErrorException: can not find service io.netifi.proteus.contract.HelloService
at io.rsocket.exceptions.Exceptions.from(Exceptions.java:53)
at io.rsocket.RSocketClient.handleFrame(RSocketClient.java:464)
at io.rsocket.RSocketClient.handleIncomingFrames(RSocketClient.java:427)
at reactor.core.publisher.LambdaSubscriber.onNext(LambdaSubscriber.java:130)
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onNext(MonoFlatMapMany.java:238)
at reactor.core.publisher.FluxGroupBy$UnicastGroupedFlux.drainRegular(FluxGroupBy.java:554)
at reactor.core.publisher.FluxGroupBy$UnicastGroupedFlux.drain(FluxGroupBy.java:630)
at reactor.core.publisher.FluxGroupBy$UnicastGroupedFlux.subscribe(FluxGroupBy.java:696)
at reactor.core.publisher.Flux.subscribe(Flux.java:7727)
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:184)
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1476)
at reactor.core.publisher.MonoProcessor.onNext(MonoProcessor.java:389)
at io.rsocket.internal.ClientServerInputMultiplexer.lambda$new$1(ClientServerInputMultiplexer.java:98)
at reactor.core.publisher.LambdaSubscriber.onNext(LambdaSubscriber.java:130)
at reactor.core.publisher.FluxGroupBy$GroupByMain.drainLoop(FluxGroupBy.java:380)
at reactor.core.publisher.FluxGroupBy$GroupByMain.drain(FluxGroupBy.java:316)
at reactor.core.publisher.FluxGroupBy$GroupByMain.onNext(FluxGroupBy.java:201)
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
at reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:211)
at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:327)
at reactor.netty.channel.ChannelOperations.onInboundNext(ChannelOperations.java:310)
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:141)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1429)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1199)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1243)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:628)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:563)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:442)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
… 1 common frames omitted
Suppressed: java.lang.Exception: #block terminated with an error
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:93)
at reactor.core.publisher.Mono.block(Mono.java:1474)
at io.netifi.proteus.quickstart.client.ClientRunner.run(ClientRunner.java:31)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:813)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:797)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:324)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248)
at io.netifi.proteus.quickstart.client.Main.main(Main.java:10)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
… 1 common frames omitted

Error on the service:
2019-03-11 20:15:00.607 ERROR 16922 — [actor-tcp-nio-2] i.n.p.r.WeightedReconnectingRSocket : proteus client received unhandled exception for connection with address localhost:8001

io.rsocket.rpc.exception.ServiceNotFound: can not find service io.netifi.proteus.contract.HelloService
        at io.rsocket.rpc.rsocket.RequestHandlingRSocket.requestResponse(RequestHandlingRSocket.java:63)
        at io.rsocket.util.RSocketProxy.requestResponse(RSocketProxy.java:40)
        at io.netifi.proteus.rsocket.AbstractUnwrappingRSocket.requestResponse(AbstractUnwrappingRSocket.java:31)
        at io.netifi.proteus.rsocket.UnwrappingRSocket.requestResponse(UnwrappingRSocket.java:10)
        at io.rsocket.RSocketServer.requestResponse(RSocketServer.java:170)
        at io.rsocket.RSocketServer.handleFrame(RSocketServer.java:250)
        at reactor.core.publisher.LambdaSubscriber.onNext(LambdaSubscriber.java:130)
        at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onNext(MonoFlatMapMany.java:238)
        at reactor.core.publisher.FluxGroupBy$UnicastGroupedFlux.drainRegular(FluxGroupBy.java:554)
        at reactor.core.publisher.FluxGroupBy$UnicastGroupedFlux.drain(FluxGroupBy.java:630)
        at reactor.core.publisher.FluxGroupBy$UnicastGroupedFlux.subscribe(FluxGroupBy.java:696)
        at reactor.core.publisher.Flux.subscribe(Flux.java:7727)
        at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:184)
        at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1476)
        at reactor.core.publisher.MonoProcessor.onNext(MonoProcessor.java:389)
        at io.rsocket.internal.ClientServerInputMultiplexer.lambda$new$1(ClientServerInputMultiplexer.java:94)
        at reactor.core.publisher.LambdaSubscriber.onNext(LambdaSubscriber.java:130)
        at reactor.core.publisher.FluxGroupBy$GroupByMain.drainLoop(FluxGroupBy.java:380)
        at reactor.core.publisher.FluxGroupBy$GroupByMain.drain(FluxGroupBy.java:316)
        at reactor.core.publisher.FluxGroupBy$GroupByMain.onNext(FluxGroupBy.java:201)
        at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
        at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)
        at reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:211)
        at reactor.netty.channel.FluxReceive.onInboundNext(FluxReceive.java:327)
        at reactor.netty.channel.ChannelOperations.onInboundNext(ChannelOperations.java:310)
        at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:141)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1429)
        at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1199)
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1243)
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:628)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:563)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:442)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
        at java.lang.Thread.run(Thread.java:748)

Versions from my pom:
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.source>1.8</maven.compiler.source>
<proteus.version>${project.version}</proteus.version>
<proteus.spring.version>1.5.3</proteus.spring.version>
<spring.boot.version>2.1.1.RELEASE</spring.boot.version>
<rsocket.rpc.version>0.2.12</rsocket.rpc.version>
<protobuf.version>3.6.1</protobuf.version>
2.9.1

Thanks in advance.


#2

Hey @WillC I believe the issue might be stemming from the fact that the service and client can’t seem to connect to a running broker on your laptop?

This error indicates to me that the service tried connecting to a broker running on localhost:8001, and wasn’t able to establish a socket connection.

If you have docker installed, this command should start up a broker for you:

docker run \
-p 8001:8001 \
-p 7001:7001 \
-p 9000:9000 \
-e BROKER_SERVER_OPTS="'-Dnetifi.authentication.0.accessKey=9007199254740991'  \
'-Dnetifi.broker.console.enabled=true' \
'-Dnetifi.authentication.0.accessToken=kTBDVtfRBO4tHOnZzSyY5ym2kfY=' \
'-Dnetifi.broker.admin.accessKey=9007199254740991' \
'-Dnetifi.broker.admin.accessToken=kTBDVtfRBO4tHOnZzSyY5ym2kfY='" \
netifi/proteus:1.5.3

#3

Hey Alan thanks for the reply. I do believe I have a running broker, which I run using that same command (we generated some new keys/tokens but I made sure to change those in the application.properties).

It’s strange because when I run the service, I see it in the broker logs and on the web interface, yet it still gives me that error.


#4

@WillC do you have your additional service code in GitHub somewhere so that I can try to reproduce?


#5

Just to share what my initial intuition, I think the issue might be that both service instances are joining the same group, defined in application.properties netifi.proteus.group=quickstart.services.helloservices

The broker essentially has a 50/50 chance to route a client to either of those and assumes that all members of that group support the same set of services. So a client for io.netifi.proteus.contract.HelloService may end up calling a service instance that only supports io.netifi.proteus.quickstart.service.HelloService and it will throw the exception that you saw.


#6

@kyle In my own code I have one service hooked up to Postgres and another to MongoDB, so as a sanity check I cloned the spring quickstart code again and tried to make 2 services again, making sure they were in different service groups. It worked when I pretty much copy pasted the original service. Once I changed the protobuf package/namespace, it said it could no longer find the service again giving the same error. Are package names etc tied to the Proteus service group names or services in some way?

Here’s the code that shows what I mean, note that the auth token/keys are different from default (running using maven):


#7

@WillC

Sort of :slight_smile: I found the issue. The java package for another-service (io.netifi.proteus.quickstart.anotherhelloservice) does not match the package name in the proto file (io.netifi.proteus.quickstart). So Spring is failing to auto-wire the service implementation into the Proteus client. Locally I changed the package for the implementation and it worked like a champ. I’ll submit a PR to your repo to try out!

EDIT: Oh right, I don’t have rights to your repo… I’ll link to a fork shortly.

EDIT2: And here it is: https://github.com/wchaney98/proteus-spring-multiple-services/pull/1


#8

Thanks Kyle, worked like a charm! (and I see… more of that auto-wiring magic…) Now trying to figure out why my other project is still having this error… I’ve triple checked all the package names etc, and I wonder if it has something to do with my dependencies? I.e. I had to (mostly arbitrarily) decide on these versions here:

        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-buffer</artifactId>
            <version>4.1.29.Final</version>
        </dependency>
        <dependency>
            <groupId>io.projectreactor.netty</groupId>
            <artifactId>reactor-netty</artifactId>
            <version>0.8.1.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>io.projectreactor.addons</groupId>
            <artifactId>reactor-extra</artifactId>
            <version>3.2.0.RELEASE</version>
        </dependency>

due to some dependency conflicts between Proteus and the Reactive Postgres/MongoDB packages. These are the versions I settled on since it was the first combination that successfully compiled. Do you think this could be causing the issue or potentially other issues? I plan on starting again from scratch but would like your input before I start.

For reference here are the Postgres/MongoDB (and related) dependencies I added:

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-mongodb-reactive</artifactId>
        <version>2.1.3.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-jdbc</artifactId>
        <version>1.0.0.r2dbc-SNAPSHOT</version>
    </dependency>
    <dependency>
        <groupId>io.r2dbc</groupId>
        <artifactId>r2dbc-postgresql</artifactId>
        <version>1.0.0.ISSUE-10-SNAPSHOT</version>
    </dependency>
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>1.18.6</version>
    </dependency>

Thanks again for your help so far


#9

There shouldn’t be any conflicts with the versions you posted.


#10

@WillC have you run into any further issues with this? As Robert mentioned, there shouldn’t be any conflicts there.


#11

I’ll have time to work on this again today and keep you guys posted. :+1:


#12

Bit of a late update but so far so good, no more issues as of right now. Thanks all for the help.