Problem with Service Endpoint matching when using different network names

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem with Service Endpoint matching when using different network names

Peter Hermsdorf-2

Hi,

I need to bring up a issue that was already discussed earlier, eg.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=419744

and here

http://dev.eclipse.org/mhonarc/lists/ecf-dev/msg08047.html

The issue is, that in our setup, the service matching is not working when accessing the remote server using a different hostname than the machine uses itself (a little hard to express as non native speaker ;) see ENDPOINT definition below)


Scott's last comment on the problem was to get more information about our use case and environment so I'll try to provide the required information

* we have a lot of clients using remote services from one application server

* we use the ecf generic provider.

* the remote service "discovery" is done by generating an EndpointDescription on client side, as all clients know by configuration which server to speak to. We use something like this to define the service endpoint:

props.put(ENDPOINT_ID, "ecftcp://" + host + ":8889/server");

where host is the hostname or IP address under which the client can access the remote server - which is not necessarily the hostname or IP which is configured at the server itself (which is exactly the issue). So eg. the server is exporting the service under endpoint

ecftcp://myhostname.company.local:8889/server

whereas the client needs to access the server eg. by using

ecftcp://otherhostname.public.net:8889/server or by ecftcp://192.168.0.100:8889/server

as endpoint id.

and even case sensitivity does matter: Some Windows machines (for whatever reason) have hostnames like MYHOSTNAME.company.local. Clients then are required to use the exact hostname with the correct upper/lowercase to access the server.

* the remote services are imported on client side via

remoteServiceAdmin.importService(endpointDescription);

* the remote services are exported on server side using DS with additional properties, like:

@Component(immediate = true, property = {
        "service.exported.interfaces=*",
        "service.exported.configs=ecf.generic.server",
        "ecf.generic.server.port=8889" })
public class MyserviceService implements IMyService {

* on server side we have a custom implementation of IHostContainerSelector which allows us to override

protected IRemoteServiceContainer createRSContainer

to add a custom RegistrySharedObject which customizes error handling ("exception to String serialization")


That's it i think.

So getting rid of the hostname part in the endpoint description and/or  when matching would probably solve the problem...


Thanks for any hints on this issue!

Best regards,

Peter


BTW: the search function of the ecf-dev archive does not seem to work:

http://dev.eclipse.org/mhonarc/lists/ecf-dev/

click on search results in a 404


_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Scott Lewis-2
Hi Peter,

On 6/21/2018 2:08 AM, Peter Hermsdorf wrote:

Hi,

I need to bring up a issue that was already discussed earlier, eg.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=419744

and here

http://dev.eclipse.org/mhonarc/lists/ecf-dev/msg08047.html

The issue is, that in our setup, the service matching is not working when accessing the remote server using a different hostname than the machine uses itself (a little hard to express as non native speaker ;) see ENDPOINT definition below)


Scott's last comment on the problem was to get more information about our use case and environment so I'll try to provide the required information

* we have a lot of clients using remote services from one application server

* we use the ecf generic provider.

* the remote service "discovery" is done by generating an EndpointDescription on client side, as all clients know by configuration which server to speak to. We use something like this to define the service endpoint:

props.put(ENDPOINT_ID, "ecftcp://" + host + ":8889/server");

where host is the hostname or IP address under which the client can access the remote server - which is not necessarily the hostname or IP which is configured at the server itself (which is exactly the issue). So eg. the server is exporting the service under endpoint

ecftcp://myhostname.company.local:8889/server

whereas the client needs to access the server eg. by using

ecftcp://otherhostname.public.net:8889/server or by ecftcp://192.168.0.100:8889/server

as endpoint id.

and even case sensitivity does matter: Some Windows machines (for whatever reason) have hostnames like MYHOSTNAME.company.local. Clients then are required to use the exact hostname with the correct upper/lowercase to access the server.

* the remote services are imported on client side via

remoteServiceAdmin.importService(endpointDescription);

* the remote services are exported on server side using DS with additional properties, like:

@Component(immediate = true, property = {
        "service.exported.interfaces=*",
        "service.exported.configs=ecf.generic.server",
        "ecf.generic.server.port=8889" })
public class MyserviceService implements IMyService {

* on server side we have a custom implementation of IHostContainerSelector which allows us to override

protected IRemoteServiceContainer createRSContainer

to add a custom RegistrySharedObject which customizes error handling ("exception to String serialization")


That's it i think.

So getting rid of the hostname part in the endpoint description and/or  when matching would probably solve the problem...


Thanks for any hints on this issue!



Ok, thanks for the description of the issue.   For this use case:  i.e. hostname used by clients that are different from the bind address for the server...i.e. client wants to use:

otherhostname.public.net

and the server needs to expose/listen on this interface:

myhostname.company.local  (or rather the ip address if the interface for this name)

For the generic provider, here are the remote service configuration properties

https://wiki.eclipse.org/ECF_Generic_Provider_Configuration_Properties#Remote_Service_Configuration_Properties

For the *external* address/name these are the relevant properties:

ecf.generic.server.hostname = otherhostname.public.net
ecf.generic.server.port = 8889
ecf.generic.server.path = /server

for example, these values will result in an external name of:  ecftcp://otherhostname.public.net:8889/server.  For the generic provider the name of the container and the external name must match.

There is also a service property:

ecf.generic.server.bindAddress = InetAddress.getByName("myhostname.company.local")

or InetAddress.getByName("13.91.95.74")...with your actual IP.

The bindAddress property ultimately specifies the *address that is passed to this ServerSocket constructor to listen on for the ecftcp server:

new ServerSocket(port, backlog, bindAddress)

https://docs.oracle.com/javase/7/docs/api/java/net/ServerSocket.html#ServerSocket(int,%20int,%20java.net.InetAddress)

So I think the solution for your use case is to use all four of these service properties set appropriately...e.g.:

ecf.generic.server.hostname = otherhostname.public.net
ecf.generic.server.port = 8889
ecf.generic.server.path = /server
ecf.generic.server.bindAddress = InetAddress.getByName("myhostname.company.local")

Since I don't currently have access to a dual-homed server I haven't been able to test this fully, but really the only thing it does is to pass the bindAddress to the ServerSocket, rather than using the default ('0.0.0.0')...which is all interfaces.

I realize now that the use of the InetAddress instance in the service properties is a little clumsy...it would be better if it could also take a String...e.g.:

ecf.generic.server.bindAddress=myhostname.company.local

If the above works for you/your use case, I'll add this (support of String as well as InetAddress) in a new enhancement for 3.14.1.

BTW...since you are overriding the HostContainerSelector for another purpose, you could pass the bindAddress (and the other properties explicitly by overriding AbstractContainerSelector.createContainer, or AbstractContainerSelector.getContainerFactoryArguments.   Be a little careful with this, however, as these methods are responsible for doing the RSA-specified handling of service properties (e.g. the ecf.generic.server. prefix is stripped off before the hostname, port, path, bindAddress are passed to the container instantiator as per RSA).

Another way to accomplish the above would be to also use the RemoteServiceAdmin on the server-side (in addition to the import/client side).   i.e. you can explicitly export a service with:

RemoteServiceAdmin.exportService(ServiceReference,Map<String,?> overridingProperties);

The exportService is usually done by the BasicTopologyManager when a remote service is registered with a call to:

RemoteServiceAdmin.exportService(ref, null)

but you can do it yourself programmatically at any time after registration via the exportService.   Note the overridingProperties...that allows you to set any properties you want (e.g. your four and the required service.exported.interfaces) that will be used to configure the server container, but then it doesn't require any properties set via @Component.   This might be more convenient for you/your use case, since you may not want to have all of the properties and their values hard coded in the @Component metadata.   With exportService you can also do things like use config admin to config your exports rather than putting app-specific info in the @Component properties.

The more that I think about it, the less I think you should look to override the HostContainerSelector...and rather use RemoteServiceAdmin.exportService if you need/want to do things programmatically on the export side.  

Best regards,

Peter


BTW: the search function of the ecf-dev archive does not seem to work:

http://dev.eclipse.org/mhonarc/lists/ecf-dev/

click on search results in a 404


Thanks...this appears to be some sort of EF infrastructure problem...I've opened this bug:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=536151

Scott





_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev



_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Peter Hermsdorf-2

Hi Scott,


thanks for your suggestions. I will need some days to try them out.


Am 21.06.2018 um 19:14 schrieb Scott Lewis:
Since I don't currently have access to a dual-homed server I haven't been able to test this fully, but really the only thing it does is to pass the bindAddress to the ServerSocket, rather than using the default ('0.0.0.0')...which is all interfaces.
Another way (or another use case) to "test" this issue:
* get VirtualBox and fire up a virtual machine
* start the server part inside the virtual machine (this leads to an endpoint id of the server with the hostname of the virtual machine)
* setup a port forwarding in VirtualBox from the host machine to the virtual machine (in my example port 8889 from host to port 8889 of the virtual machine)
* now try to connect to the server part in the virtual machine from the host machine using "localhost" as hostname in the endpoint description

-> leads to a network connection from client to server but without working remote service

Thanks again!

Bye Peter

_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Peter Hermsdorf-2
In reply to this post by Scott Lewis-2

Hi Scott,

thanks for your suggestions.

I'm afraid to say that this does not solve all issues for us.

If i understand you correct, you were describing different ways to change the service endpoint on the server to match the one that the client uses to connect to the exported service on the server.

The problem is, we have setups where different client machines use different network names to connect to the same server instance. e.g.. clients in the local network of the server can use a local DNS to connect to the server by a locally known host name. We have other clients connecting via a remote network (VPN tunnel) and sometimes we can't use the local DNS setup trough that tunnel, but just the IP address or a completely different hostname for network configuration reasons.

I think what we need is a completely hostname (and port) agnostic way to match the service endpoints between the client and the server.

In other words: one thing is to define the network target (by hostname or IP and port) and the other thing would be to match the remote service once the client and server are connected by network (via the service name and version etc.).

I'm willing to implement some code to do that, if you could point me in the right direction....

Thanks!

Bye Peter


Am 21.06.2018 um 19:14 schrieb Scott Lewis:
Hi Peter,

On 6/21/2018 2:08 AM, Peter Hermsdorf wrote:

Hi,

I need to bring up a issue that was already discussed earlier, eg.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=419744

and here

http://dev.eclipse.org/mhonarc/lists/ecf-dev/msg08047.html

The issue is, that in our setup, the service matching is not working when accessing the remote server using a different hostname than the machine uses itself (a little hard to express as non native speaker ;) see ENDPOINT definition below)


Scott's last comment on the problem was to get more information about our use case and environment so I'll try to provide the required information

* we have a lot of clients using remote services from one application server

* we use the ecf generic provider.

* the remote service "discovery" is done by generating an EndpointDescription on client side, as all clients know by configuration which server to speak to. We use something like this to define the service endpoint:

props.put(ENDPOINT_ID, "ecftcp://" + host + ":8889/server");

where host is the hostname or IP address under which the client can access the remote server - which is not necessarily the hostname or IP which is configured at the server itself (which is exactly the issue). So eg. the server is exporting the service under endpoint

ecftcp://myhostname.company.local:8889/server

whereas the client needs to access the server eg. by using

ecftcp://otherhostname.public.net:8889/server or by ecftcp://192.168.0.100:8889/server

as endpoint id.

and even case sensitivity does matter: Some Windows machines (for whatever reason) have hostnames like MYHOSTNAME.company.local. Clients then are required to use the exact hostname with the correct upper/lowercase to access the server.

* the remote services are imported on client side via

remoteServiceAdmin.importService(endpointDescription);

* the remote services are exported on server side using DS with additional properties, like:

@Component(immediate = true, property = {
        "service.exported.interfaces=*",
        "service.exported.configs=ecf.generic.server",
        "ecf.generic.server.port=8889" })
public class MyserviceService implements IMyService {

* on server side we have a custom implementation of IHostContainerSelector which allows us to override

protected IRemoteServiceContainer createRSContainer

to add a custom RegistrySharedObject which customizes error handling ("exception to String serialization")


That's it i think.

So getting rid of the hostname part in the endpoint description and/or  when matching would probably solve the problem...


Thanks for any hints on this issue!



Ok, thanks for the description of the issue.   For this use case:  i.e. hostname used by clients that are different from the bind address for the server...i.e. client wants to use:

otherhostname.public.net

and the server needs to expose/listen on this interface:

myhostname.company.local  (or rather the ip address if the interface for this name)

For the generic provider, here are the remote service configuration properties

https://wiki.eclipse.org/ECF_Generic_Provider_Configuration_Properties#Remote_Service_Configuration_Properties

For the *external* address/name these are the relevant properties:

ecf.generic.server.hostname = otherhostname.public.net
ecf.generic.server.port = 8889
ecf.generic.server.path = /server

for example, these values will result in an external name of:  ecftcp://otherhostname.public.net:8889/server.  For the generic provider the name of the container and the external name must match.

There is also a service property:

ecf.generic.server.bindAddress = InetAddress.getByName("myhostname.company.local")

or InetAddress.getByName("13.91.95.74")...with your actual IP.

The bindAddress property ultimately specifies the *address that is passed to this ServerSocket constructor to listen on for the ecftcp server:

new ServerSocket(port, backlog, bindAddress)

https://docs.oracle.com/javase/7/docs/api/java/net/ServerSocket.html#ServerSocket(int,%20int,%20java.net.InetAddress)

So I think the solution for your use case is to use all four of these service properties set appropriately...e.g.:

ecf.generic.server.hostname = otherhostname.public.net
ecf.generic.server.port = 8889
ecf.generic.server.path = /server
ecf.generic.server.bindAddress = InetAddress.getByName("myhostname.company.local")

Since I don't currently have access to a dual-homed server I haven't been able to test this fully, but really the only thing it does is to pass the bindAddress to the ServerSocket, rather than using the default ('0.0.0.0')...which is all interfaces.

I realize now that the use of the InetAddress instance in the service properties is a little clumsy...it would be better if it could also take a String...e.g.:

ecf.generic.server.bindAddress=myhostname.company.local

If the above works for you/your use case, I'll add this (support of String as well as InetAddress) in a new enhancement for 3.14.1.

BTW...since you are overriding the HostContainerSelector for another purpose, you could pass the bindAddress (and the other properties explicitly by overriding AbstractContainerSelector.createContainer, or AbstractContainerSelector.getContainerFactoryArguments.   Be a little careful with this, however, as these methods are responsible for doing the RSA-specified handling of service properties (e.g. the ecf.generic.server. prefix is stripped off before the hostname, port, path, bindAddress are passed to the container instantiator as per RSA).

Another way to accomplish the above would be to also use the RemoteServiceAdmin on the server-side (in addition to the import/client side).   i.e. you can explicitly export a service with:

RemoteServiceAdmin.exportService(ServiceReference,Map<String,?> overridingProperties);

The exportService is usually done by the BasicTopologyManager when a remote service is registered with a call to:

RemoteServiceAdmin.exportService(ref, null)

but you can do it yourself programmatically at any time after registration via the exportService.   Note the overridingProperties...that allows you to set any properties you want (e.g. your four and the required service.exported.interfaces) that will be used to configure the server container, but then it doesn't require any properties set via @Component.   This might be more convenient for you/your use case, since you may not want to have all of the properties and their values hard coded in the @Component metadata.   With exportService you can also do things like use config admin to config your exports rather than putting app-specific info in the @Component properties.

The more that I think about it, the less I think you should look to override the HostContainerSelector...and rather use RemoteServiceAdmin.exportService if you need/want to do things programmatically on the export side.  

Best regards,

_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Scott Lewis-2
In reply to this post by Peter Hermsdorf-2
Hi Peter

Im out of town until next week and will respond properly then

Scott

From: "Peter Hermsdorf" &lt;[hidden email]&gt;
Date: Fri, Jun 29, 2018 12:09 am
To: [hidden email]
Subject: Re: [ecf-dev] Problem with Service Endpoint matching when using different network names
Hi Scott,
thanks for your suggestions.
I'm afraid to say that this does not solve all issues for us.
If i understand you correct, you were describing different ways to change the service endpoint on the server to match the one that the client uses to connect to the exported service on the server.
The problem is, we have setups where different client machines use different network names to connect to the same server instance. e.g.. clients in the local network of the server can use a local DNS to connect to the server by a locally known host name. We have other clients connecting via a remote network (VPN tunnel) and sometimes we can't use the local DNS setup trough that tunnel, but just the IP address or a completely different hostname for network configuration reasons.
I think what we need is a completely hostname (and port) agnostic way to match the service endpoints between the client and the server.
 
In other words: one thing is to define the network target (by hostname or IP and port) and the other thing would be to match the remote service once the client and server are connected by network (via the service name and version etc.).
 
I'm willing to implement some code to do that, if you could point me in the right direction....
Thanks! Bye Peter

 

 

 

 Am 21.06.2018 um 19:14 schrieb Scott Lewis:
 
 
 
 
 
  Hi Peter,
 
 
 
 On 6/21/2018 2:08 AM, Peter Hermsdorf wrote:
 
 
 
 
  Hi,
  I need to bring up a issue that was already discussed earlier, eg.
 
  https://bugs.eclipse.org/bugs/show_bug.cgi?id=419744 
  and here
 
  http://dev.eclipse.org/mhonarc/lists/ecf-dev/msg08047.html 
  The issue is, that in our setup, the service matching is not working when accessing the remote server using a different hostname than the machine uses itself (a little hard to express as non native speaker ;) see ENDPOINT definition below)
 
 
  Scott's last comment on the problem was to get more information about our use case and environment so I'll try to provide the required information
 
  * we have a lot of clients using remote services from one application server
  * we use the ecf generic provider.
 
  * the remote service "discovery" is done by generating an EndpointDescription on client side, as all clients know by configuration which server to speak to. We use something like this to define the service endpoint:
  props.put(ENDPOINT_ID, "ecftcp://" + host + ":8889/server");
  where host is the hostname or IP address under which the client can access the remote server - which is not necessarily the hostname or IP which is configured at the server itself (which is exactly the issue). So eg. the server is exporting the service under endpoint
  ecftcp://myhostname.company.local:8889/server
  whereas the client needs to access the server eg. by using
  ecftcp://otherhostname.public.net:8889/server or by ecftcp://192.168.0.100:8889/server
  as endpoint id.
 
  and even case sensitivity does matter: Some Windows machines (for whatever reason) have hostnames like MYHOSTNAME.company.local. Clients then are required to use the exact hostname with the correct upper/lowercase to access the server.
  * the remote services are imported on client side via
  remoteServiceAdmin.importService(endpointDescription);
 
  * the remote services are exported on server side using DS with additional properties, like:
  @Component(immediate = true, property = {
   "service.exported.interfaces=*",
   "service.exported.configs=ecf.generic.server",
   "ecf.generic.server.port=8889" })
 public class MyserviceService implements IMyService {
  * on server side we have a custom implementation of IHostContainerSelector which allows us to override
  protected IRemoteServiceContainer createRSContainer
 
  to add a custom RegistrySharedObject which customizes error handling ("exception to String serialization")
 
 
  That's it i think.
 
  So getting rid of the hostname part in the endpoint description and/or when matching would probably solve the problem...
 
 
 
  Thanks for any hints on this issue!
 
 
 
 
 Ok, thanks for the description of the issue. For this use case: i.e. hostname used by clients that are different from the bind address for the server...i.e. client wants to use:
 
 
 
 otherhostname.public.net
 
 
 
 and the server needs to expose/listen on this interface:
 
 
 
 myhostname.company.local (or rather the ip address if the interface for this name)
 
 
 
 For the generic provider, here are the remote service configuration properties
 
 
 
 
 https://wiki.eclipse.org/ECF_Generic_Provider_Configuration_Properties#Remote_Service_Configuration_Properties
 
 
 
 For the *external* address/name these are the relevant properties:
 
 
 
 ecf.generic.server.hostname = otherhostname.public.net
 
 ecf.generic.server.port = 8889
 
 ecf.generic.server.path = /server
 
 
 
 for example, these values will result in an external name of: ecftcp://otherhostname.public.net:8889/server. For the generic provider the name of the container and the external name must match.
 
 
 
 There is also a service property:
 
 
 
 ecf.generic.server.bindAddress = InetAddress.getByName("myhostname.company.local")
 
 
 
 or InetAddress.getByName("13.91.95.74")...with your actual IP.
 
 
 
 The bindAddress property ultimately specifies the *address that is passed to this ServerSocket constructor to listen on for the ecftcp server:
 
 
 
 new ServerSocket(port, backlog, bindAddress)
 
 
 
 
 https://docs.oracle.com/javase/7/docs/api/java/net/ServerSocket.html#ServerSocket(int,%20int,%20java.net.InetAddress)
 
 
 
 So I think the solution for your use case is to use all four of these service properties set appropriately...e.g.:
 
 
 
 ecf.generic.server.hostname = otherhostname.public.net
 
 ecf.generic.server.port = 8889
 
 ecf.generic.server.path = /server
 
 ecf.generic.server.bindAddress = InetAddress.getByName("myhostname.company.local")
 
 
 
 Since I don't currently have access to a dual-homed server I haven't been able to test this fully, but really the only thing it does is to pass the bindAddress to the ServerSocket, rather than using the default ('0.0.0.0')...which is all interfaces.
 
 
 
 I realize now that the use of the InetAddress instance in the service properties is a little clumsy...it would be better if it could also take a String...e.g.:
 
 
 
 ecf.generic.server.bindAddress=myhostname.company.local
 
 
 
 If the above works for you/your use case, I'll add this (support of String as well as InetAddress) in a new enhancement for 3.14.1.
 
 
 
 BTW...since you are overriding the HostContainerSelector for another purpose, you could pass the bindAddress (and the other properties explicitly by overriding AbstractContainerSelector.createContainer, or AbstractContainerSelector.getContainerFactoryArguments. Be a little careful with this, however, as these methods are responsible for doing the RSA-specified handling of service properties (e.g. the ecf.generic.server. prefix is stripped off before the hostname, port, path, bindAddress are passed to the container instantiator as per RSA).
 
 
 
 Another way to accomplish the above would be to also use the RemoteServiceAdmin on the server-side (in addition to the import/client side). i.e. you can explicitly export a service with:
 
 
 
 RemoteServiceAdmin.exportService(ServiceReference,Map&lt;String,?&gt; overridingProperties);
 
 
 
 The exportService is usually done by the BasicTopologyManager when a remote service is registered with a call to:
 
 
 
 RemoteServiceAdmin.exportService(ref, null)
 
 
 
 but you can do it yourself programmatically at any time after registration via the exportService. Note the overridingProperties...that allows you to set any properties you want (e.g. your four and the required service.exported.interfaces) that will be used to configure the server container, but then it doesn't require any properties set via @Component. This might be more convenient for you/your use case, since you may not want to have all of the properties and their values hard coded in the @Component metadata. With exportService you can also do things like use config admin to config your exports rather than putting app-specific info in the @Component properties.
 
 
 
 The more that I think about it, the less I think you should look to override the HostContainerSelector...and rather use RemoteServiceAdmin.exportService if you need/want to do things programmatically on the export side.
 
 
 
 Best regards,
 
 
 
 
   
   
   
   
 

_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Peter Hermsdorf-2

Hi Scott,

just a short ping on this issue. Would be great if you could manage to look at my issue.

Thanks!

bye Peter

Am 29.06.2018 um 20:25 schrieb Scott Lewis:
Hi Peter

Im out of town until next week and will respond properly then

Scott

From: "Peter Hermsdorf" &lt;[hidden email]&gt;
Date: Fri, Jun 29, 2018 12:09 am
To: [hidden email]
Subject: Re: [ecf-dev] Problem with Service Endpoint matching when using different network names
Hi Scott, 
thanks for your suggestions. 
I'm afraid to say that this does not solve all issues for us. 
If i understand you correct, you were describing different ways to change the service endpoint on the server to match the one that the client uses to connect to the exported service on the server. 
The problem is, we have setups where different client machines use different network names to connect to the same server instance. e.g.. clients in the local network of the server can use a local DNS to connect to the server by a locally known host name. We have other clients connecting via a remote network (VPN tunnel) and sometimes we can't use the local DNS setup trough that tunnel, but just the IP address or a completely different hostname for network configuration reasons. 
I think what we need is a completely hostname (and port) agnostic way to match the service endpoints between the client and the server. 
  
In other words: one thing is to define the network target (by hostname or IP and port) and the other thing would be to match the remote service once the client and server are connected by network (via the service name and version etc.).
  
I'm willing to implement some code to do that, if you could point me in the right direction.... 
Thanks! Bye Peter

 

 

 

 Am 21.06.2018 um 19:14 schrieb Scott Lewis:
 
 
 
  
 
  Hi Peter,
  
 
  
 On 6/21/2018 2:08 AM, Peter Hermsdorf wrote:
  
 
  
  
  Hi, 
  I need to bring up a issue that was already discussed earlier, eg. 
  
  https://bugs.eclipse.org/bugs/show_bug.cgi?id=419744 
  and here
  
  http://dev.eclipse.org/mhonarc/lists/ecf-dev/msg08047.html 
  The issue is, that in our setup, the service matching is not working when accessing the remote server using a different hostname than the machine uses itself (a little hard to express as non native speaker ;) see ENDPOINT definition below) 
  
  
  Scott's last comment on the problem was to get more information about our use case and environment so I'll try to provide the required information 
  
  * we have a lot of clients using remote services from one application server 
  * we use the ecf generic provider. 
  
  * the remote service "discovery" is done by generating an EndpointDescription on client side, as all clients know by configuration which server to speak to. We use something like this to define the service endpoint: 
  props.put(ENDPOINT_ID, "ecftcp://" + host + ":8889/server"); 
  where host is the hostname or IP address under which the client can access the remote server - which is not necessarily the hostname or IP which is configured at the server itself (which is exactly the issue). So eg. the server is exporting the service under endpoint 
  ecftcp://myhostname.company.local:8889/server 
  whereas the client needs to access the server eg. by using 
  ecftcp://otherhostname.public.net:8889/server or by ecftcp://192.168.0.100:8889/server 
  as endpoint id.
  
  and even case sensitivity does matter: Some Windows machines (for whatever reason) have hostnames like MYHOSTNAME.company.local. Clients then are required to use the exact hostname with the correct upper/lowercase to access the server. 
  * the remote services are imported on client side via 
  remoteServiceAdmin.importService(endpointDescription);
  
  * the remote services are exported on server side using DS with additional properties, like: 
  @Component(immediate = true, property = {
   "service.exported.interfaces=*",
   "service.exported.configs=ecf.generic.server",
   "ecf.generic.server.port=8889" })
 public class MyserviceService implements IMyService { 
  * on server side we have a custom implementation of IHostContainerSelector which allows us to override 
  protected IRemoteServiceContainer createRSContainer 
  
  to add a custom RegistrySharedObject which customizes error handling ("exception to String serialization") 
  
  
  That's it i think. 
  
  So getting rid of the hostname part in the endpoint description and/or when matching would probably solve the problem...
  
  
  
  Thanks for any hints on this issue! 
  
 
 
 
 Ok, thanks for the description of the issue. For this use case: i.e. hostname used by clients that are different from the bind address for the server...i.e. client wants to use:
 
 
 
 otherhostname.public.net
 
 
 
 and the server needs to expose/listen on this interface:
 
 
 
 myhostname.company.local (or rather the ip address if the interface for this name)
 
 
 
 For the generic provider, here are the remote service configuration properties
 
 
 
 
 https://wiki.eclipse.org/ECF_Generic_Provider_Configuration_Properties#Remote_Service_Configuration_Properties
 
 
 
 For the *external* address/name these are the relevant properties:
 
 
 
 ecf.generic.server.hostname = otherhostname.public.net
 
 ecf.generic.server.port = 8889
 
 ecf.generic.server.path = /server
 
 
 
 for example, these values will result in an external name of: ecftcp://otherhostname.public.net:8889/server. For the generic provider the name of the container and the external name must match.
 
 
 
 There is also a service property:
 
 
 
 ecf.generic.server.bindAddress = InetAddress.getByName("myhostname.company.local") 
 
 
 
 or InetAddress.getByName("13.91.95.74")...with your actual IP.
 
 
 
 The bindAddress property ultimately specifies the *address that is passed to this ServerSocket constructor to listen on for the ecftcp server:
 
 
 
 new ServerSocket(port, backlog, bindAddress)
 
 
 
 
 https://docs.oracle.com/javase/7/docs/api/java/net/ServerSocket.html#ServerSocket(int,%20int,%20java.net.InetAddress)
 
 
 
 So I think the solution for your use case is to use all four of these service properties set appropriately...e.g.:
 
 
 
 ecf.generic.server.hostname = otherhostname.public.net
 
 ecf.generic.server.port = 8889
 
 ecf.generic.server.path = /server
 
 ecf.generic.server.bindAddress = InetAddress.getByName("myhostname.company.local") 
 
 
 
 Since I don't currently have access to a dual-homed server I haven't been able to test this fully, but really the only thing it does is to pass the bindAddress to the ServerSocket, rather than using the default ('0.0.0.0')...which is all interfaces.
 
 
 
 I realize now that the use of the InetAddress instance in the service properties is a little clumsy...it would be better if it could also take a String...e.g.:
 
 
 
 ecf.generic.server.bindAddress=myhostname.company.local
 
 
 
 If the above works for you/your use case, I'll add this (support of String as well as InetAddress) in a new enhancement for 3.14.1.
 
 
 
 BTW...since you are overriding the HostContainerSelector for another purpose, you could pass the bindAddress (and the other properties explicitly by overriding AbstractContainerSelector.createContainer, or AbstractContainerSelector.getContainerFactoryArguments. Be a little careful with this, however, as these methods are responsible for doing the RSA-specified handling of service properties (e.g. the ecf.generic.server. prefix is stripped off before the hostname, port, path, bindAddress are passed to the container instantiator as per RSA).
 
 
 
 Another way to accomplish the above would be to also use the RemoteServiceAdmin on the server-side (in addition to the import/client side). i.e. you can explicitly export a service with:
 
 
 
 RemoteServiceAdmin.exportService(ServiceReference,Map&lt;String,?&gt; overridingProperties);
 
 
 
 The exportService is usually done by the BasicTopologyManager when a remote service is registered with a call to:
 
 
 
 RemoteServiceAdmin.exportService(ref, null)
 
 
 
 but you can do it yourself programmatically at any time after registration via the exportService. Note the overridingProperties...that allows you to set any properties you want (e.g. your four and the required service.exported.interfaces) that will be used to configure the server container, but then it doesn't require any properties set via @Component. This might be more convenient for you/your use case, since you may not want to have all of the properties and their values hard coded in the @Component metadata. With exportService you can also do things like use config admin to config your exports rather than putting app-specific info in the @Component properties.
 
 
 
 The more that I think about it, the less I think you should look to override the HostContainerSelector...and rather use RemoteServiceAdmin.exportService if you need/want to do things programmatically on the export side. 
 
 
 
 Best regards,

_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Scott Lewis-2
In reply to this post by Peter Hermsdorf-2
Hi Peter,

On 6/29/2018 12:09 AM, Peter Hermsdorf wrote:

>
> Hi Scott,
>
> thanks for your suggestions.
>
> I'm afraid to say that this does not solve all issues for us.
>
> If i understand you correct, you were describing different ways to
> change the service endpoint on the server to match the one that the
> client uses to connect to the exported service on the server.
>
> The problem is, we have setups where different client machines use
> different network names to connect to the same server instance.
>
| e.g.. clients in the local network of the server can use a local DNS
to connect to the server by a locally known host name. We have other
clients connecting via a remote network (VPN tunnel) and sometimes we
can't use the local DNS setup trough that tunnel, but just the IP
address or a completely different hostname for network configuration
reasons.


I see.   So external clients are using VPN addressing and internal
clients LAN addresses.   Correct?   And you would like a single
server(s) (with 1...or multiple nics?), to host remote service(s) to
both types of clients....internal and external.  Something like this:

C1-->addr:foo1.bar.com:1234-->VPN-->IP1 addr of
server1-->Server1-->Serviceinstance
                                                         |
                    C2-> IP1 (or IP2?) addr of server1---|

Seem right?

For the moment, forget about the dns names.   What do the IP
addresses/port look like for your C1 and C2?   e.g. what does C1, C2
server address look like, what are IP1 and IP2, does the Server1 have
multiple nics (IP1/IP2) or just one?


> I think what we need is a completely hostname (and port) agnostic way
> to match the service endpoints between the client and the server.
>
> In other words: one thing is to define the network target (by hostname
> or IP and port) and the other thing would be to match the remote
> service once the client and server are connected by network (via the
> service name and version etc.).
>

Not sure if this is what you are suggesting...but there could be a
remote service to do that matching between client and endpoint, I suppose.

Modifying a single generic provider to do this could be difficult. The
main reason for this is that the generic provider has the notion of a
group of connected processes... .the server's unique group name must
consist of *some* address that is understood as unique among all group
members connected within a particularly container instance ...e.g.
ecftcp://foo1.bar.com:3282/server is the 'name/ID' of the server in the
group connected that container instance (and it's unique ID).

However, it's quite possible to export a single service with multiple
generic provider container instances...e.g. couldn't you export service
with a container for VPN clients, and another that exports the same
service using 'internal' addressing (with different IP and port)?

As for other distribution providers

It's possible that R0SGi could work for this, but the VPN might still
present a problem.

You could use one of the http-based providers (e.g. Jersey or CXF) in
combination with the generic provider...e.g. you could export with both
generic and http-based distribution providers simultaneously to expose
the remote service via two (or more) different transports.

Also:   Is it possible to consider alternative network topologies? e.g.
having both an external and internal servers?  or running the service
outside vpn/in cloud?

We can continue to chat here, or if you would like contact me directly
at slewis at composent.com or scottslewis at gmail.com. If we can work
something out, I could assist your development or do custom development
or both.

> I'm willing to implement some code to do that, if you could point me
> in the right direction....
>

I'm happy to point you in the right direction, but I have a feeling that
there could be multiple ways to deal with this, so there might not be
just one 'right direction'.   Let's keep discussing.

Scott

_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Peter Hermsdorf-2
Hi,

sorry for being late with my answer. See my comments inline.
I see.   So external clients are using VPN addressing and internal clients LAN addresses.   Correct?   And you would like a single server(s) (with 1...or multiple nics?), to host remote service(s) to both types of clients....internal and external.  Something like this:

C1-->addr:foo1.bar.com:1234-->VPN-->IP1 addr of server1-->Server1-->Serviceinstance
                                                        |
                   C2-> IP1 (or IP2?) addr of server1---|

Seem right?
Correct
For the moment, forget about the dns names.   What do the IP addresses/port look like for your C1 and C2?   e.g. what does C1, C2 server address look like, what are IP1 and IP2, does the Server1 have multiple nics (IP1/IP2) or just one?
In most installations the server has one IP address. Sometimes we can use this IP address when accessing the server trough VPN, sometimes there is a mapping somewhere in between...

I think what we need is a completely hostname (and port) agnostic way to match the service endpoints between the client and the server.

In other words: one thing is to define the network target (by hostname or IP and port) and the other thing would be to match the remote service once the client and server are connected by network (via the service name and version etc.).

Modifying a single generic provider to do this could be difficult. The main reason for this is that the generic provider has the notion of a group of connected processes... .the server's unique group name must consist of *some* address that is understood as unique among all group members connected within a particularly container instance ...e.g. ecftcp://foo1.bar.com:3282/server is the 'name/ID' of the server in the group connected that container instance (and it's unique ID).
from my point of view a unique ID for a remote service could well be ecftcp://server .
But the challenge here is probably to split the "do the network connection" code from "match the correct service" code, since as far as I can tell the endpoint ID (ecftcp://foo1.bar.com:3282/server) is used for both parts.

However, it's quite possible to export a single service with multiple generic provider container instances...e.g. couldn't you export service with a container for VPN clients, and another that exports the same service using 'internal' addressing (with different IP and port)?

As for other distribution providers

It's possible that R0SGi could work for this, but the VPN might still present a problem.

You could use one of the http-based providers (e.g. Jersey or CXF) in combination with the generic provider...e.g. you could export with both generic and http-based distribution providers simultaneously to expose the remote service via two (or more) different transports.
I would like to avoid to having different ways/options/configuration to connect to remote services since that would mean a more or less complicated configuration for developers and consultants connecting to remote services running at the customer

Also:   Is it possible to consider alternative network topologies? e.g. having both an external and internal servers?  or running the service outside vpn/in cloud?
No, all installations are on premise at the customers
I'm happy to point you in the right direction, but I have a feeling that there could be multiple ways to deal with this, so there might not be just one 'right direction'.   Let's keep discussing.
I'm open to use/adapt a different distribution provider when that solves the problem.

My main "wish" would be to decouple the "raw" network connection (use any IP that works or any hostname that works - in term of getting a network connection) and not mix that with the actual service that is then consumed/provided at that network location.

If that does not work well with the generic provider, which one would suit best and what needs to be changed?


Thanks for your hints!

Bye Peter

_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Scott Lewis-2
On 7/18/2018 4:50 AM, Peter Hermsdorf wrote:

> Hi,
>
> sorry for being late with my answer. See my comments inline.
>> I see.   So external clients are using VPN addressing and internal
>> clients LAN addresses.   Correct?   And you would like a single
>> server(s) (with 1...or multiple nics?), to host remote service(s) to
>> both types of clients....internal and external. Something like this:
>>
>> C1-->addr:foo1.bar.com:1234-->VPN-->IP1 addr of
>> server1-->Server1-->Serviceinstance
>>                                                         |
>>                    C2-> IP1 (or IP2?) addr of server1---|
>>
>> Seem right?
> Correct
>> For the moment, forget about the dns names.   What do the IP
>> addresses/port look like for your C1 and C2?   e.g. what does C1, C2
>> server address look like, what are IP1 and IP2, does the Server1 have
>> multiple nics (IP1/IP2) or just one?
> In most installations the server has one IP address. Sometimes we can
> use this IP address when accessing the server trough VPN, sometimes
> there is a mapping somewhere in between...

When the mapping is done, who/what does the mapping?..and how is it
done?   It seems to me that's the problematic case.  Does it map both IP
address and port, or just IP address?   Do you know if it's using NAT or
some other tech?

Are there other services on this net...e.g. a web server...that are
working properly with the addressing properties that you are looking
for?   If so, how is that done?   A reverse proxy or load balancing hw or ?


>>
>>> I think what we need is a completely hostname (and port) agnostic
>>> way to match the service endpoints between the client and the server.
>>>
>>> In other words: one thing is to define the network target (by
>>> hostname or IP and port) and the other thing would be to match the
>>> remote service once the client and server are connected by network
>>> (via the service name and version etc.).
>>
>> Modifying a single generic provider to do this could be difficult.
>> The main reason for this is that the generic provider has the notion
>> of a group of connected processes... .the server's unique group name
>> must consist of *some* address that is understood as unique among all
>> group members connected within a particularly container instance
>> ...e.g. ecftcp://foo1.bar.com:3282/server is the 'name/ID' of the
>> server in the group connected that container instance (and it's
>> unique ID).
> from my point of view a unique ID for a remote service could well be
> ecftcp://server .

Unfortunately it can't be that, because if more than one server used
that same identifier and a single client was trying to get to two
different service instances there would be no way to tell them apart.

> But the challenge here is probably to split the "do the network
> connection" code from "match the correct service" code, since as far
> as I can tell the endpoint ID (ecftcp://foo1.bar.com:3282/server) is
> used for both parts.

To be clear, even with the generic provider there is a separation
between the id identifying the java process that contains remote
services (ecftcp://foo1.bar.com:3282/server) and the remote service
instance:  (i.e. 1, 2, 3...(long).  In the edef the container-relative
service instance is the property with name: ecf.rsvc.id.

But the real problem is that since the generic provider has the notion
of a connected group of processes (containers)...i.e. peer-to-peer,
rather than a simple client -> server structure, it's necessary to give
the group manager (the server with a hub and spoke topology) a unique id
within the group that all clients can agree on.   For tcp-based comm
(generic provider), that id is usually the host:port combination
(InetAddress) since that's guaranteed globally unique and doesn't
require any additional name mapping.    However, on the network you are
targeting it seems that some clients need to use IP address 1 and other
clients IP address 2...to access the same server and service.

I suspect that your network topology may require moving to a strict
client -> server notion of rpc, rather than using a group topology.   It
would mean that all services would be exported by a server and imported
by clients, and that no services would be exported by clients or
imported by server.

If having a strict client->server works for your services, then I would
suggest you try the either the JaxRSRemoteService providers [1] which
are based upon HttpService (jetty server usually).   It still seems to
me that you would need a reverse proxy like nginx to expose the same
server to access via multiple IP addresses/networks, and I'm not sure if
that's possible on your target network, but nginx is frequently used for
that.

With strict client-server it's possible that ROSGi (based upon tcp)
would also work, but since I don't yet understand what your address
mapping is about (e.g. NAT, etc), and I haven't tried ROSGI under
anything other than 'normal' addressing/networking scenarios, I don't
feel comfortable suggesting that.   It also seems to me that a tcp
reverse proxy might be needed also (a relatively recent nginx feature).

>
>> However, it's quite possible to export a single service with multiple
>> generic provider container instances...e.g. couldn't you export
>> service with a container for VPN clients, and another that exports
>> the same service using 'internal' addressing (with different IP and
>> port)?
>>
>> As for other distribution providers
>>
>> It's possible that R0SGi could work for this, but the VPN might still
>> present a problem.
>>
>> You could use one of the http-based providers (e.g. Jersey or CXF) in
>> combination with the generic provider...e.g. you could export with
>> both generic and http-based distribution providers simultaneously to
>> expose the remote service via two (or more) different transports.
> I would like to avoid to having different ways/options/configuration
> to connect to remote services since that would mean a more or less
> complicated configuration for developers and consultants connecting to
> remote services running at the customer

If I'm understanding you right, you will at least need to provide
different target IP addresses/port combinations to different clients C1
and C2, right?   I don't see a way to avoid that, unless you provide
some other public service that does that...i.e. analogous to DNS but
just for your app.   You could do that, btw without too much
difficulty...i.e. create a public/internet etcd server, for example
(etcd is the http-based discovery protocol that Google uses for
Kubernetes).   It would require running yet another server (etcd),
maintaining, etc...but at least you don't have to develop it.

>
>> Also: Is it possible to consider alternative network topologies? e.g.
>> having both an external and internal servers?  or running the service
>> outside vpn/in cloud?
> No, all installations are on premise at the customers
>> I'm happy to point you in the right direction, but I have a feeling
>> that there could be multiple ways to deal with this, so there might
>> not be just one 'right direction'.   Let's keep discussing.
> I'm open to use/adapt a different distribution provider when that
> solves the problem.

I would try strict client->server over http first.   Actually I would
suggest just running small http server on your server process...with a
single servlet or even just page, and if you can configure things to
access in the way you want via C1 and C2...CN, then you almost certainly
will be able to use osgi, jetty + one of the JaxRSRemoteService providers.

>
> My main "wish" would be to decouple the "raw" network connection (use
> any IP that works or any hostname that works - in term of getting a
> network connection) and not mix that with the actual service that is
> then consumed/provided at that network location.

Yes.  I think if you can adopt a client -> server/service model for all
your services, and can get a combination of tech (e.g. reverse proxy +
http server) that allows http access with the desired addressing over
your target networks, then I expect you can use one of the JaxRS
distribution providers without too much trouble.   It will likely
require changing the serialization of method args and return values.

For ECF Photon I've done quite a lot of work on the JaxRS distribution
providers (e.g. to move them up to OSGi R7 spec and to support both
Jersey and CXF underneath), but I've tried to make them extensible and
configurable (e.g. using Jaxrs extensions) and will provide direct
support for these/all providers as I can.

Scott

[1] https://github.com/ECF/JaxRSProviders
[2] https://wiki.eclipse.org/Tutorial:_JaxRS_Remote_Services_on_Karaf

_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Peter Hermsdorf-2

Hi Scott,


Am 21.07.2018 um 00:11 schrieb Scott Lewis:
When the mapping is done, who/what does the mapping?..and how is it done?   It seems to me that's the problematic case.  Does it map both IP address and port, or just IP address?   Do you know if it's using NAT or some other tech?

Are there other services on this net...e.g. a web server...that are working properly with the addressing properties that you are looking for?   If so, how is that done?   A reverse proxy or load balancing hw or ?
Typically mapping is only done for the ip addresses. Actually i can't tell exactly how it's done. Probably it's custom to every customer....
Other services that work that way are JDBC connections to oracle databases. They don't care if you reach them by hostname or the one ip address or the other ... ;)


from my point of view a unique ID for a remote service could well be ecftcp://server .

Unfortunately it can't be that, because if more than one server used that same identifier and a single client was trying to get to two different service instances there would be no way to tell them apart.

But the challenge here is probably to split the "do the network connection" code from "match the correct service" code, since as far as I can tell the endpoint ID (ecftcp://foo1.bar.com:3282/server) is used for both parts.


agree on.   For tcp-based comm (generic provider), that id is usually the host:port combination (InetAddress) since that's guaranteed globally unique and doesn't require any additional name mapping.    However, on the network you are targeting it seems that some clients need to use IP address 1 and other clients IP address 2...to access the same server and service.

I suspect that your network topology may require moving to a strict client -> server notion of rpc, rather than using a group topology.   It would mean that all services would be exported by a server and imported by clients, and that no services would be exported by clients or imported by server.

1)  currently we only use "strict" client->server setup, but: i could image use-cases where it could be useful if the server could import services from the clients to realize something like push information to the client from the server (without polling etc)  - but that's probably another story...
If having a strict client->server works for your services, then I would suggest you try the either the JaxRSRemoteService providers [1] which are based upon HttpService (jetty server usually).   It still seems to me that you would need a reverse proxy like nginx to expose the same server to access via multiple IP addresses/networks, and I'm not sure if that's possible on your target network, but nginx is frequently used for that.

2) i don't think it's a good idea to switch to a http based communication - performance wise. i would like to stick with a binary transportation layer rather than have http protocol overhead (remember my kryo serialization implementation)... and we don't have a use case where other services/participants would benefit from a http based communication...
we have a jetty on server side, but it has nothing to deal with the osgi remote services - just provides some jax-rs rest services
With strict client-server it's possible that ROSGi (based upon tcp) would also work, but since I don't yet understand what your address mapping is about (e.g. NAT, etc), and I haven't tried ROSGI under anything other than 'normal' addressing/networking scenarios, I don't feel comfortable suggesting that.   It also seems to me that a tcp reverse proxy might be needed also (a relatively recent nginx feature).

If I'm understanding you right, you will at least need to provide different target IP addresses/port combinations to different clients C1 and C2, right?   I don't see a way to avoid that, unless you provide some other public service that does that...i.e. analogous to DNS but just for your app.   You could do that, btw without too much difficulty...i.e. create a public/internet etcd server, for example (etcd is the http-based discovery protocol that Google uses for Kubernetes).   It would require running yet another server (etcd), maintaining, etc...but at least you don't have to develop it.

3) switching to a different provider is an option, if there is "no" performance problem and this "connection" issue would be solved. additional infrastructure for translating/mapping/proxying is a problem and is at the end no real option.... from my point of view that's the job of the underlying tcp/ip network...

Because of the many questions regarding how the network mapping etc. is done i would like to describe another scenario which shows the same problem, but is probably better reproducible:

Use virtualbox on your host machine and install a virtual machine (e.g. running linux). let's name it server1. deploy a service with the generic provider on server1. the service will bind to the local network interface and use the local name of the linux machine: server1. that means e.g. the endpoint ID is ecftcp://server1.local:3282/server .

in virtualbox on the host machine configure a port mapping from port 3282 into the virtual machine with the same port 3282.

from a network perspective you are now able to reach the service in the linux box from your host machine using tcp localhost:3282.

if you now start a service consumer on your host machine which uses the endpoint ecftcp://localhost:3282/server or the real hostname of the host machine e.g. ecftcp://scotts-machine:3282/server you will get a succesful tcp/ip connection between client and server, but (of course) the service import is not working because of the different endpoint id's ....

i need a solution for that without adding local hostname entries, dns changes and additional servers or infrastructure ;)

sorry to bug you with this issue that long ;)

Thanks again for your comments and hints!

Best regards,
Peter




_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Scott Lewis-2
Hi Peter,

On 7/26/2018 1:41 AM, Peter Hermsdorf wrote:

Hi Scott,


Am 21.07.2018 um 00:11 schrieb Scott Lewis:
When the mapping is done, who/what does the mapping?..and how is it done?   It seems to me that's the problematic case.  Does it map both IP address and port, or just IP address?   Do you know if it's using NAT or some other tech?

Are there other services on this net...e.g. a web server...that are working properly with the addressing properties that you are looking for?   If so, how is that done?   A reverse proxy or load balancing hw or ?
Typically mapping is only done for the ip addresses. Actually i can't tell exactly how it's done. Probably it's custom to every customer....
Other services that work that way are JDBC connections to oracle databases. They don't care if you reach them by hostname or the one ip address or the other ... ;)

Given the smiley, I'm not sure if you are joking but here's the rub:   JDBC connections are point-to-point (strict client-server)...and so the addressing is relatively simple.   Part of its simplicity is that it creates isolation between clients, but with DB connections that's generally what you want.

However, the ECF generic provider...and some of the others...provides a group model, where every process (clients and server) can both export and import remote services as opposed to server-export and client-import only.

Just to explain a little more:   In a group model (i.e. ECF container), every process in the group has to

a) have a unique identity;
b) agree (membership) to use the same ID to refer to the same process.  

So in the three group members case:

Serverid -> ecftcp://some.name:3333/server

Client1id -> 1

Client2id -> 2

When the two clients connect to this server, all three processes receive the IDs of the other two processes...i.e. Server gets 1, 2, Client1 gets serverid, 2; and Client2 gets serverid and 1.    Note that if the Client1 serverid != Client2 serverid (i.e. the clients are on different networks) then it violates b above.    This seems to be your situation with the generic provider.

All I'm saying is that the introduction of NAT, firewalls, proxies, VPNs, etc changes the addressing.   This is less of a problem for client-server communication because there are only two processes 'aware' of each other instead of a group.

When I wrote the generic provider (originally > 14 years ago), the addressing introduced by NAT, VPN, etc wasn't nearly as prevalent.   I *could* have introduced some additional connection/group join protocol to associate some separate/unique name (uuid, etc) with the server ip address, so that clients didn't use some.name:3333.   However, at that time I didn't anticipate it would be necessary, and so I didn't do that...using the (guaranteed unique) some.name:3333 to both identify the process and client uses to connect to the server.  In retrospect it would have been nice if I did, but OTOH given the complexity involved in doing it in the 'general case' I'm kind of glad I didn't :).  

It's possible that a new/extended generic container could be created that had this additional protocol to have connected clients use a non-ip-based name for the server in the group.   It would probably be necessary, however, to first understand what the name mapping was doing for a given network topology (i.e. your customer) at least if one was interested in keeping the 'group' nature (i.e. not be 'strict client-server' at the service level).    

As we've been discussing, another option is to use a strict client-server topology rather than a group, and use or create a distribution provider based upon a strict client-server model.   See below for more comments about this.

<stuff deleted>

1)  currently we only use "strict" client->server setup, but: i could image use-cases where it could be useful if the server could import services from the clients to realize something like push information to the client from the server (without polling etc)  - but that's probably another story...

Indeed it is :).

If having a strict client->server works for your services, then I would suggest you try the either the JaxRSRemoteService providers [1] which are based upon HttpService (jetty server usually).   It still seems to me that you would need a reverse proxy like nginx to expose the same server to access via multiple IP addresses/networks, and I'm not sure if that's possible on your target network, but nginx is frequently used for that.

2) i don't think it's a good idea to switch to a http based communication - performance wise. i would like to stick with a binary transportation layer rather than have http protocol overhead (remember my kryo serialization implementation)... and we don't have a use case where other services/participants would benefit from a http based communication...
we have a jetty on server side, but it has nothing to deal with the osgi remote services - just provides some jax-rs rest services

Given that you already have a jetty server working in this topology, perhaps it would be worth it to give it a try with the JaxRSProvider...and see how the performance is for a test service.   I understand the concern about performance with http...especially if you are sending lots of messages.   However, as you know jetty/websockets, caching proxies, hw, etc., etc have improved the performance of http under many usage scenarios so maybe it will be less of a problem than you think.

Another thought:   Once you were confident that a strict client-server model would get you want you want in terms of connection, you could create a simple websocket-or-regular-socket-based distribution provider based upon your Kyro serialization provider [1] or at least starting from that.   With Photon I've tried to make it easier to create new distribution providers (more/more useful abstract classes).   There are a other providers at [2] that you could model from (e.g. Chronicle, grcp, etc) or just use a simple socket connection based upon the trivial provider [3].

You can also combine multiple distribution providers if you need to (i.e. some services with JaxRS, others with a custom-socket-based distribution provider for others).

<stuff deleted>

3) switching to a different provider is an option, if there is "no" performance problem and this "connection" issue would be solved. additional infrastructure for translating/mapping/proxying is a problem and is at the end no real option.... from my point of view that's the job of the underlying tcp/ip network...

That would be very nice, but unfortunately these days with NAT, VPN, etc we are not dealing with just one tcp/ip network :).


Because of the many questions regarding how the network mapping etc. is done i would like to describe another scenario which shows the same problem, but is probably better reproducible:

Thanks, this is helpful.


Use virtualbox on your host machine and install a virtual machine (e.g. running linux). let's name it server1. deploy a service with the generic provider on server1. the service will bind to the local network interface and use the local name of the linux machine: server1. that means e.g. the endpoint ID is ecftcp://server1.local:3282/server .

in virtualbox on the host machine configure a port mapping from port 3282 into the virtual machine with the same port 3282.

from a network perspective you are now able to reach the service in the linux box from your host machine using tcp localhost:3282.

if you now start a service consumer on your host machine which uses the endpoint ecftcp://localhost:3282/server or the real hostname of the host machine e.g. ecftcp://scotts-machine:3282/server you will get a succesful tcp/ip connection between client and server, but (of course) the service import is not working because of the different endpoint id's ....

Right.   See explanation above for why this is the case with a group model/generic distribution provider.

So, if you can give up the group model and the ability to export services from peers...and it sounds like you can...then you should be able to give one of the JaxRSProviders a try.

If tcp is needed for your required performance, then to be safe, I would suggest trying a very very small Java application to create a socket connection, read and write a few bytes and make sure that your target network will allow such client-server comm, as many VPN networks limit traffic by port and/or protocol, etc.  This is one reason it's so hard for me or anyone to create a 'general' socket provider that will work on any VPN, NAT, network, etc.  They can be configured in ways that will allow some things (e.g. odbc, http over specific ports) and not allow others (e.g. socket comm over port xxxx).

If the Java socket app works in your target environment then I would suggest creating a very simple new distribution provider starting with the trivial provider [3].   If you decide to do this I can/will help and would welcome it, but for full-effort on it I would need some additional arrangement.


i need a solution for that without adding local hostname entries, dns changes and additional servers or infrastructure ;)

Ok.   

Regards,

Scott

[1] https://github.com/ECF/kryo-serialization
[2] https://github.com/ECF
[3] http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/tree/examples/bundles/org.eclipse.ecf.examples.provider.trivial


_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Scott Lewis-2
Hi Peter,

I couldn't resist.   I've created a TCP socket-based distribution provider [1].   Like we've been discussing, it's client->server only, and so clients cannot export services (and servers cannot import services), but unlike the other example impls using tcp that I'm aware of, it supports multiple remote calls via a single connection, and it supports multiple remote services exported via a single socket listener.   It also supports the OSGi R7 async remote services and the other R7 features (intents, configurable timeout, etc).   And it just uses ServerSocket and Socket classes, so there's a chance it will work in your target environment (if the environment allows non-known apps to make direct socket connections with an unknown...not JDBC, not http, etc protocol.

Of course it's not done and fully tested yet, but I did get signs of life this evening.

Scott

[1] https://github.com/ECF/tcpsocketdistributionprovider

On 7/26/2018 10:52 AM, Scott Lewis wrote:
Hi Peter,

On 7/26/2018 1:41 AM, Peter Hermsdorf wrote:

Hi Scott,


Am 21.07.2018 um 00:11 schrieb Scott Lewis:
When the mapping is done, who/what does the mapping?..and how is it done?   It seems to me that's the problematic case.  Does it map both IP address and port, or just IP address?   Do you know if it's using NAT or some other tech?

Are there other services on this net...e.g. a web server...that are working properly with the addressing properties that you are looking for?   If so, how is that done?   A reverse proxy or load balancing hw or ?
Typically mapping is only done for the ip addresses. Actually i can't tell exactly how it's done. Probably it's custom to every customer....
Other services that work that way are JDBC connections to oracle databases. They don't care if you reach them by hostname or the one ip address or the other ... ;)

Given the smiley, I'm not sure if you are joking but here's the rub:   JDBC connections are point-to-point (strict client-server)...and so the addressing is relatively simple.   Part of its simplicity is that it creates isolation between clients, but with DB connections that's generally what you want.

However, the ECF generic provider...and some of the others...provides a group model, where every process (clients and server) can both export and import remote services as opposed to server-export and client-import only.

Just to explain a little more:   In a group model (i.e. ECF container), every process in the group has to

a) have a unique identity;
b) agree (membership) to use the same ID to refer to the same process.  

So in the three group members case:

Serverid -> ecftcp://some.name:3333/server

Client1id -> 1

Client2id -> 2

When the two clients connect to this server, all three processes receive the IDs of the other two processes...i.e. Server gets 1, 2, Client1 gets serverid, 2; and Client2 gets serverid and 1.    Note that if the Client1 serverid != Client2 serverid (i.e. the clients are on different networks) then it violates b above.    This seems to be your situation with the generic provider.

All I'm saying is that the introduction of NAT, firewalls, proxies, VPNs, etc changes the addressing.   This is less of a problem for client-server communication because there are only two processes 'aware' of each other instead of a group.

When I wrote the generic provider (originally > 14 years ago), the addressing introduced by NAT, VPN, etc wasn't nearly as prevalent.   I *could* have introduced some additional connection/group join protocol to associate some separate/unique name (uuid, etc) with the server ip address, so that clients didn't use some.name:3333.   However, at that time I didn't anticipate it would be necessary, and so I didn't do that...using the (guaranteed unique) some.name:3333 to both identify the process and client uses to connect to the server.  In retrospect it would have been nice if I did, but OTOH given the complexity involved in doing it in the 'general case' I'm kind of glad I didn't :).  

It's possible that a new/extended generic container could be created that had this additional protocol to have connected clients use a non-ip-based name for the server in the group.   It would probably be necessary, however, to first understand what the name mapping was doing for a given network topology (i.e. your customer) at least if one was interested in keeping the 'group' nature (i.e. not be 'strict client-server' at the service level).    

As we've been discussing, another option is to use a strict client-server topology rather than a group, and use or create a distribution provider based upon a strict client-server model.   See below for more comments about this.

<stuff deleted>

1)  currently we only use "strict" client->server setup, but: i could image use-cases where it could be useful if the server could import services from the clients to realize something like push information to the client from the server (without polling etc)  - but that's probably another story...

Indeed it is :).

If having a strict client->server works for your services, then I would suggest you try the either the JaxRSRemoteService providers [1] which are based upon HttpService (jetty server usually).   It still seems to me that you would need a reverse proxy like nginx to expose the same server to access via multiple IP addresses/networks, and I'm not sure if that's possible on your target network, but nginx is frequently used for that.

2) i don't think it's a good idea to switch to a http based communication - performance wise. i would like to stick with a binary transportation layer rather than have http protocol overhead (remember my kryo serialization implementation)... and we don't have a use case where other services/participants would benefit from a http based communication...
we have a jetty on server side, but it has nothing to deal with the osgi remote services - just provides some jax-rs rest services

Given that you already have a jetty server working in this topology, perhaps it would be worth it to give it a try with the JaxRSProvider...and see how the performance is for a test service.   I understand the concern about performance with http...especially if you are sending lots of messages.   However, as you know jetty/websockets, caching proxies, hw, etc., etc have improved the performance of http under many usage scenarios so maybe it will be less of a problem than you think.

Another thought:   Once you were confident that a strict client-server model would get you want you want in terms of connection, you could create a simple websocket-or-regular-socket-based distribution provider based upon your Kyro serialization provider [1] or at least starting from that.   With Photon I've tried to make it easier to create new distribution providers (more/more useful abstract classes).   There are a other providers at [2] that you could model from (e.g. Chronicle, grcp, etc) or just use a simple socket connection based upon the trivial provider [3].

You can also combine multiple distribution providers if you need to (i.e. some services with JaxRS, others with a custom-socket-based distribution provider for others).

<stuff deleted>

3) switching to a different provider is an option, if there is "no" performance problem and this "connection" issue would be solved. additional infrastructure for translating/mapping/proxying is a problem and is at the end no real option.... from my point of view that's the job of the underlying tcp/ip network...

That would be very nice, but unfortunately these days with NAT, VPN, etc we are not dealing with just one tcp/ip network :).


Because of the many questions regarding how the network mapping etc. is done i would like to describe another scenario which shows the same problem, but is probably better reproducible:

Thanks, this is helpful.


Use virtualbox on your host machine and install a virtual machine (e.g. running linux). let's name it server1. deploy a service with the generic provider on server1. the service will bind to the local network interface and use the local name of the linux machine: server1. that means e.g. the endpoint ID is ecftcp://server1.local:3282/server .

in virtualbox on the host machine configure a port mapping from port 3282 into the virtual machine with the same port 3282.

from a network perspective you are now able to reach the service in the linux box from your host machine using tcp localhost:3282.

if you now start a service consumer on your host machine which uses the endpoint ecftcp://localhost:3282/server or the real hostname of the host machine e.g. ecftcp://scotts-machine:3282/server you will get a succesful tcp/ip connection between client and server, but (of course) the service import is not working because of the different endpoint id's ....

Right.   See explanation above for why this is the case with a group model/generic distribution provider.

So, if you can give up the group model and the ability to export services from peers...and it sounds like you can...then you should be able to give one of the JaxRSProviders a try.

If tcp is needed for your required performance, then to be safe, I would suggest trying a very very small Java application to create a socket connection, read and write a few bytes and make sure that your target network will allow such client-server comm, as many VPN networks limit traffic by port and/or protocol, etc.  This is one reason it's so hard for me or anyone to create a 'general' socket provider that will work on any VPN, NAT, network, etc.  They can be configured in ways that will allow some things (e.g. odbc, http over specific ports) and not allow others (e.g. socket comm over port xxxx).

If the Java socket app works in your target environment then I would suggest creating a very simple new distribution provider starting with the trivial provider [3].   If you decide to do this I can/will help and would welcome it, but for full-effort on it I would need some additional arrangement.


i need a solution for that without adding local hostname entries, dns changes and additional servers or infrastructure ;)

Ok.   

Regards,

Scott

[1] https://github.com/ECF/kryo-serialization
[2] https://github.com/ECF
[3] http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/tree/examples/bundles/org.eclipse.ecf.examples.provider.trivial



_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev



_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Peter Hermsdorf-2

Hi Scott,

that sounds and looks really promising! I'll need some time to process the information you gave and look at the code and then come back to you.

Have a nice weekend!

Bye Peter


Am 27.07.2018 um 03:54 schrieb Scott Lewis:
Hi Peter,

I couldn't resist.   I've created a TCP socket-based distribution provider [1].   Like we've been discussing, it's client->server only, and so clients cannot export services (and servers cannot import services), but unlike the other example impls using tcp that I'm aware of, it supports multiple remote calls via a single connection, and it supports multiple remote services exported via a single socket listener.   It also supports the OSGi R7 async remote services and the other R7 features (intents, configurable timeout, etc).   And it just uses ServerSocket and Socket classes, so there's a chance it will work in your target environment (if the environment allows non-known apps to make direct socket connections with an unknown...not JDBC, not http, etc protocol.

Of course it's not done and fully tested yet, but I did get signs of life this evening.

Scott

[1] https://github.com/ECF/tcpsocketdistributionprovider

On 7/26/2018 10:52 AM, Scott Lewis wrote:
Hi Peter,

On 7/26/2018 1:41 AM, Peter Hermsdorf wrote:

Hi Scott,


Am 21.07.2018 um 00:11 schrieb Scott Lewis:
When the mapping is done, who/what does the mapping?..and how is it done?   It seems to me that's the problematic case.  Does it map both IP address and port, or just IP address?   Do you know if it's using NAT or some other tech?

Are there other services on this net...e.g. a web server...that are working properly with the addressing properties that you are looking for?   If so, how is that done?   A reverse proxy or load balancing hw or ?
Typically mapping is only done for the ip addresses. Actually i can't tell exactly how it's done. Probably it's custom to every customer....
Other services that work that way are JDBC connections to oracle databases. They don't care if you reach them by hostname or the one ip address or the other ... ;)

Given the smiley, I'm not sure if you are joking but here's the rub:   JDBC connections are point-to-point (strict client-server)...and so the addressing is relatively simple.   Part of its simplicity is that it creates isolation between clients, but with DB connections that's generally what you want.

However, the ECF generic provider...and some of the others...provides a group model, where every process (clients and server) can both export and import remote services as opposed to server-export and client-import only.

Just to explain a little more:   In a group model (i.e. ECF container), every process in the group has to

a) have a unique identity;
b) agree (membership) to use the same ID to refer to the same process.  

So in the three group members case:

Serverid -> ecftcp://some.name:3333/server

Client1id -> 1

Client2id -> 2

When the two clients connect to this server, all three processes receive the IDs of the other two processes...i.e. Server gets 1, 2, Client1 gets serverid, 2; and Client2 gets serverid and 1.    Note that if the Client1 serverid != Client2 serverid (i.e. the clients are on different networks) then it violates b above.    This seems to be your situation with the generic provider.

All I'm saying is that the introduction of NAT, firewalls, proxies, VPNs, etc changes the addressing.   This is less of a problem for client-server communication because there are only two processes 'aware' of each other instead of a group.

When I wrote the generic provider (originally > 14 years ago), the addressing introduced by NAT, VPN, etc wasn't nearly as prevalent.   I *could* have introduced some additional connection/group join protocol to associate some separate/unique name (uuid, etc) with the server ip address, so that clients didn't use some.name:3333.   However, at that time I didn't anticipate it would be necessary, and so I didn't do that...using the (guaranteed unique) some.name:3333 to both identify the process and client uses to connect to the server.  In retrospect it would have been nice if I did, but OTOH given the complexity involved in doing it in the 'general case' I'm kind of glad I didn't :).  

It's possible that a new/extended generic container could be created that had this additional protocol to have connected clients use a non-ip-based name for the server in the group.   It would probably be necessary, however, to first understand what the name mapping was doing for a given network topology (i.e. your customer) at least if one was interested in keeping the 'group' nature (i.e. not be 'strict client-server' at the service level).    

As we've been discussing, another option is to use a strict client-server topology rather than a group, and use or create a distribution provider based upon a strict client-server model.   See below for more comments about this.

<stuff deleted>

1)  currently we only use "strict" client->server setup, but: i could image use-cases where it could be useful if the server could import services from the clients to realize something like push information to the client from the server (without polling etc)  - but that's probably another story...

Indeed it is :).

If having a strict client->server works for your services, then I would suggest you try the either the JaxRSRemoteService providers [1] which are based upon HttpService (jetty server usually).   It still seems to me that you would need a reverse proxy like nginx to expose the same server to access via multiple IP addresses/networks, and I'm not sure if that's possible on your target network, but nginx is frequently used for that.

2) i don't think it's a good idea to switch to a http based communication - performance wise. i would like to stick with a binary transportation layer rather than have http protocol overhead (remember my kryo serialization implementation)... and we don't have a use case where other services/participants would benefit from a http based communication...
we have a jetty on server side, but it has nothing to deal with the osgi remote services - just provides some jax-rs rest services

Given that you already have a jetty server working in this topology, perhaps it would be worth it to give it a try with the JaxRSProvider...and see how the performance is for a test service.   I understand the concern about performance with http...especially if you are sending lots of messages.   However, as you know jetty/websockets, caching proxies, hw, etc., etc have improved the performance of http under many usage scenarios so maybe it will be less of a problem than you think.

Another thought:   Once you were confident that a strict client-server model would get you want you want in terms of connection, you could create a simple websocket-or-regular-socket-based distribution provider based upon your Kyro serialization provider [1] or at least starting from that.   With Photon I've tried to make it easier to create new distribution providers (more/more useful abstract classes).   There are a other providers at [2] that you could model from (e.g. Chronicle, grcp, etc) or just use a simple socket connection based upon the trivial provider [3].

You can also combine multiple distribution providers if you need to (i.e. some services with JaxRS, others with a custom-socket-based distribution provider for others).

<stuff deleted>

3) switching to a different provider is an option, if there is "no" performance problem and this "connection" issue would be solved. additional infrastructure for translating/mapping/proxying is a problem and is at the end no real option.... from my point of view that's the job of the underlying tcp/ip network...

That would be very nice, but unfortunately these days with NAT, VPN, etc we are not dealing with just one tcp/ip network :).


Because of the many questions regarding how the network mapping etc. is done i would like to describe another scenario which shows the same problem, but is probably better reproducible:

Thanks, this is helpful.


Use virtualbox on your host machine and install a virtual machine (e.g. running linux). let's name it server1. deploy a service with the generic provider on server1. the service will bind to the local network interface and use the local name of the linux machine: server1. that means e.g. the endpoint ID is ecftcp://server1.local:3282/server .

in virtualbox on the host machine configure a port mapping from port 3282 into the virtual machine with the same port 3282.

from a network perspective you are now able to reach the service in the linux box from your host machine using tcp localhost:3282.

if you now start a service consumer on your host machine which uses the endpoint ecftcp://localhost:3282/server or the real hostname of the host machine e.g. ecftcp://scotts-machine:3282/server you will get a succesful tcp/ip connection between client and server, but (of course) the service import is not working because of the different endpoint id's ....

Right.   See explanation above for why this is the case with a group model/generic distribution provider.

So, if you can give up the group model and the ability to export services from peers...and it sounds like you can...then you should be able to give one of the JaxRSProviders a try.

If tcp is needed for your required performance, then to be safe, I would suggest trying a very very small Java application to create a socket connection, read and write a few bytes and make sure that your target network will allow such client-server comm, as many VPN networks limit traffic by port and/or protocol, etc.  This is one reason it's so hard for me or anyone to create a 'general' socket provider that will work on any VPN, NAT, network, etc.  They can be configured in ways that will allow some things (e.g. odbc, http over specific ports) and not allow others (e.g. socket comm over port xxxx).

If the Java socket app works in your target environment then I would suggest creating a very simple new distribution provider starting with the trivial provider [3].   If you decide to do this I can/will help and would welcome it, but for full-effort on it I would need some additional arrangement.


i need a solution for that without adding local hostname entries, dns changes and additional servers or infrastructure ;)

Ok.   

Regards,

Scott

[1] https://github.com/ECF/kryo-serialization
[2] https://github.com/ECF
[3] http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/tree/examples/bundles/org.eclipse.ecf.examples.provider.trivial


_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Scott Lewis-2
On 7/27/2018 7:22 AM, Peter Hermsdorf wrote:
>
> Hi Scott,
>
> that sounds and looks really promising! I'll need some time to process
> the information you gave and look at the code and then come back to you.
>

This morning I've added some code to allow properties to set the
hostname and port on server side, and to use the ecf.endpoint.id on
client side to connect to the given hostname and port.   This is
working...see TCPSocketServerContainer and TCPSocketClientContainer classes.

I've been testing localhost with the LAN-based discovery (zeroconf) so
the hostname/port exported in the endpoint description is what is used
on the client.   If/when other discovery and import is used on the
client, then it can do whatever it wants wrt what hostname and port to
use to connect to the server.

There are some ways that this is limited, but it can be generalized or
extended, and I'm hoping it will give a good idea of how to create others.

Scott

_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Peter Hermsdorf-2

Hi Scott,

thanks again for providing this sample implementation! That makes it very transparent how things work.
Actually I thought the ecf-generic provider would work that way ;)

I wasn't able to get the server up and running.  Maybe a problem with the launch configuration. See below for the bundles and their states.
I added a sample service which should by exported by adding something like

@Component(property = {
        "service.exported.interfaces=*",
        "service.exported.configs=ecf.namespace.socket"
         })
public class HelloService implements IHello{

    public HelloService() {
        System.err.println();
    }
   
    @Override
    public void sayHello() {
        System.err.println("Hello");
    }

}

but as said before, the server does not seem to get started. The start code in the Activator does the registration but the  RemoteServiceContainerInstantiator.createInstance is never called.
Any idea?

Thanks!

Bye Peter

ss

id    State       Bundle
0    ACTIVE      org.eclipse.osgi_3.12.50.v20170928-1321
                Fragments=22
2    ACTIVE      org.eclipse.equinox.common_3.9.0.v20170207-1454
5    ACTIVE      org.eclipse.ecf.remoteservice.asyncproxy_2.1.0.v20180409-2248
6    ACTIVE      javax.inject_1.0.0.v20091030
8    ACTIVE      org.eclipse.equinox.event_1.4.0.v20170105-1446
9    ACTIVE      javax.annotation_1.2.0.v201602091430
11    ACTIVE      org.eclipse.osgi.util_3.4.0.v20170111-1608
14    ACTIVE      com.ibm.icu_58.2.0.v20170418-1837
18    ACTIVE      org.eclipse.equinox.concurrent_1.1.0.v20130327-1442
19    ACTIVE      org.eclipse.osgi.services_3.6.0.v20170228-1906
22    RESOLVED    org.eclipse.osgi.compatibility.state_1.1.0.v20170516-1513
                Master=0
23    ACTIVE      org.eclipse.equinox.preferences_3.7.0.v20170126-2132
25    ACTIVE      org.apache.batik.css_1.8.0.v20170214-1941
26    ACTIVE      org.eclipse.core.contenttype_3.6.0.v20170207-1037
27    ACTIVE      org.eclipse.core.jobs_3.9.2.v20171030-1027
31    ACTIVE      org.apache.felix.scr_2.0.10.v20170501-2007
33    ACTIVE      org.eclipse.ecf.identity_3.9.0.v20180426-1936
34    ACTIVE      org.eclipse.equinox.registry_3.7.0.v20170222-1344
41    ACTIVE      org.w3c.css.sac_1.3.1.v200903091627
43    ACTIVE      org.eclipse.ecf.provider.tcpsocket.common_1.0.0.qualifier
48    ACTIVE      org.apache.commons.jxpath_1.3.0.v200911051830
50    ACTIVE      org.eclipse.ecf_3.9.0.v20180402-2015
                Fragments=52
51    ACTIVE      org.apache.batik.util_1.8.0.v20170214-1941
52    RESOLVED    org.eclipse.ecf.ssl_1.2.100.v20180301-0132
                Master=50
58    ACTIVE      org.eclipse.core.runtime_3.13.0.v20170207-1030
68    ACTIVE      org.eclipse.ecf.provider.tcpsocket.server_1.0.0.qualifier
70    ACTIVE      org.eclipse.ecf.remoteservice_8.13.0.v20180406-1915
73    ACTIVE      org.eclipse.equinox.app_1.3.400.v20150715-1528
75    ACTIVE      org.eclipse.ecf.provider.remoteservice_4.4.100.v20180516-2213
76    ACTIVE      org.eclipse.ecf.provider_4.8.0.v20180402-2103
77    ACTIVE      org.eclipse.ecf.sharedobject_2.6.0.v20180404-2345
78    ACTIVE      org.apache.felix.gogo.runtime_0.10.0.v201209301036
79    ACTIVE      org.apache.felix.gogo.shell_0.10.0.v201212101605
80    ACTIVE      org.eclipse.equinox.console_1.1.300.v20170512-2111
81    ACTIVE      service.api_1.0.0.qualifier
82    ACTIVE      service.provider_1.0.0.qualifier
83    ACTIVE      org.eclipse.ecf.discovery_5.0.300.v20180306-0211
84    ACTIVE      org.eclipse.ecf.osgi.services.distribution_2.1.200.v20180301-0016
85    ACTIVE      org.eclipse.ecf.osgi.services.remoteserviceadmin.proxy_1.0.100.v20180301-0016
86    ACTIVE      org.eclipse.ecf.osgi.services.remoteserviceadmin_4.6.800.v20180518-0149
87    ACTIVE      org.eclipse.ecf.provider.jmdns_4.3.200.v20180306-0429
88    ACTIVE      org.eclipse.osgi.services.remoteserviceadmin_1.6.200.v20180301-0016
89    ACTIVE      org.eclipse.ecf.remoteservice.eventadmin_1.3.0.v20180426-2032
90    ACTIVE      org.eclipse.ecf.server_2.1.200.v20180302-2006
91    ACTIVE      org.eclipse.equinox.ds_1.5.0.v20170307-1429


Am 27.07.2018 um 20:08 schrieb Scott Lewis:
On 7/27/2018 7:22 AM, Peter Hermsdorf wrote:

Hi Scott,

that sounds and looks really promising! I'll need some time to process the information you gave and look at the code and then come back to you.


This morning I've added some code to allow properties to set the hostname and port on server side, and to use the ecf.endpoint.id on client side to connect to the given hostname and port.   This is working...see TCPSocketServerContainer and TCPSocketClientContainer classes.

I've been testing localhost with the LAN-based discovery (zeroconf) so the hostname/port exported in the endpoint description is what is used on the client.   If/when other discovery and import is used on the client, then it can do whatever it wants wrt what hostname and port to use to connect to the server.

There are some ways that this is limited, but it can be generalized or extended, and I'm hoping it will give a good idea of how to create others.

Scott


_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Scott Lewis-2
In reply to this post by Peter Hermsdorf-2
Hi Peter

Sorry about the slow reply.  I'm out on a gig.

I think the service.exported.configs prop value should be

ecf.socket.server

As per
https://github.com/ECF/tcpsocketdistributionprovider/blob/master/bundles/org.eclipse.ecf.provider.tcpsocket.common/src/org/eclipse/ecf/provider/tcpsocket/common/TCPSocketConstants.java

RE the genetic provider...it was done long before rsa...and there is the peer services also as discussed.

Hope this helps.

Scott

From: "Peter Hermsdorf" &lt;[hidden email]&gt;
Date: Tue, Aug 7, 2018 7:07 am
To: [hidden email]
Subject: Re: [ecf-dev] Problem with Service Endpoint matching when using different network names
Hi Scott,
  thanks again for providing this sample implementation! That makes it very transparent how things work.

 Actually I thought the ecf-generic provider would work that way ;)

 

 I wasn't able to get the server up and running. Maybe a problem with the launch configuration. See below for the bundles and their states.

 I added a sample service which should by exported by adding something like

 

 @Component(property = {

   "service.exported.interfaces=*",

   "service.exported.configs=ecf.namespace.socket"

   })

 public class HelloService implements IHello{

 

  public HelloService() {

   System.err.println();

  }

 

  @Override

  public void sayHello() {

   System.err.println("Hello");

  }

 

 }

 

 but as said before, the server does not seem to get started. The start code in the Activator does the registration but the RemoteServiceContainerInstantiator.createInstance is never called.

 Any idea?

 

 Thanks!

 

 Bye Peter

 

 ss

 

 id State Bundle

 0 ACTIVE org.eclipse.osgi_3.12.50.v20170928-1321

   Fragments=22

 2 ACTIVE org.eclipse.equinox.common_3.9.0.v20170207-1454

 5 ACTIVE org.eclipse.ecf.remoteservice.asyncproxy_2.1.0.v20180409-2248

 6 ACTIVE javax.inject_1.0.0.v20091030

 8 ACTIVE org.eclipse.equinox.event_1.4.0.v20170105-1446

 9 ACTIVE javax.annotation_1.2.0.v201602091430

 11 ACTIVE org.eclipse.osgi.util_3.4.0.v20170111-1608

 14 ACTIVE com.ibm.icu_58.2.0.v20170418-1837

 18 ACTIVE org.eclipse.equinox.concurrent_1.1.0.v20130327-1442

 19 ACTIVE org.eclipse.osgi.services_3.6.0.v20170228-1906

 22 RESOLVED org.eclipse.osgi.compatibility.state_1.1.0.v20170516-1513

   Master=0

 23 ACTIVE org.eclipse.equinox.preferences_3.7.0.v20170126-2132

 25 ACTIVE org.apache.batik.css_1.8.0.v20170214-1941

 26 ACTIVE org.eclipse.core.contenttype_3.6.0.v20170207-1037

 27 ACTIVE org.eclipse.core.jobs_3.9.2.v20171030-1027

 31 ACTIVE org.apache.felix.scr_2.0.10.v20170501-2007

 33 ACTIVE org.eclipse.ecf.identity_3.9.0.v20180426-1936

 34 ACTIVE org.eclipse.equinox.registry_3.7.0.v20170222-1344

 41 ACTIVE org.w3c.css.sac_1.3.1.v200903091627

 43 ACTIVE org.eclipse.ecf.provider.tcpsocket.common_1.0.0.qualifier

 48 ACTIVE org.apache.commons.jxpath_1.3.0.v200911051830

 50 ACTIVE org.eclipse.ecf_3.9.0.v20180402-2015

   Fragments=52

 51 ACTIVE org.apache.batik.util_1.8.0.v20170214-1941

 52 RESOLVED org.eclipse.ecf.ssl_1.2.100.v20180301-0132

   Master=50

 58 ACTIVE org.eclipse.core.runtime_3.13.0.v20170207-1030

 68 ACTIVE org.eclipse.ecf.provider.tcpsocket.server_1.0.0.qualifier

 70 ACTIVE org.eclipse.ecf.remoteservice_8.13.0.v20180406-1915

 73 ACTIVE org.eclipse.equinox.app_1.3.400.v20150715-1528

 75 ACTIVE org.eclipse.ecf.provider.remoteservice_4.4.100.v20180516-2213

 76 ACTIVE org.eclipse.ecf.provider_4.8.0.v20180402-2103

 77 ACTIVE org.eclipse.ecf.sharedobject_2.6.0.v20180404-2345

 78 ACTIVE org.apache.felix.gogo.runtime_0.10.0.v201209301036

 79 ACTIVE org.apache.felix.gogo.shell_0.10.0.v201212101605

 80 ACTIVE org.eclipse.equinox.console_1.1.300.v20170512-2111

 81 ACTIVE service.api_1.0.0.qualifier

 82 ACTIVE service.provider_1.0.0.qualifier

 83 ACTIVE org.eclipse.ecf.discovery_5.0.300.v20180306-0211

 84 ACTIVE org.eclipse.ecf.osgi.services.distribution_2.1.200.v20180301-0016

 85 ACTIVE org.eclipse.ecf.osgi.services.remoteserviceadmin.proxy_1.0.100.v20180301-0016

 86 ACTIVE org.eclipse.ecf.osgi.services.remoteserviceadmin_4.6.800.v20180518-0149

 87 ACTIVE org.eclipse.ecf.provider.jmdns_4.3.200.v20180306-0429

 88 ACTIVE org.eclipse.osgi.services.remoteserviceadmin_1.6.200.v20180301-0016

 89 ACTIVE org.eclipse.ecf.remoteservice.eventadmin_1.3.0.v20180426-2032

 90 ACTIVE org.eclipse.ecf.server_2.1.200.v20180302-2006

 91 ACTIVE org.eclipse.equinox.ds_1.5.0.v20170307-1429

 

 

 

 Am 27.07.2018 um 20:08 schrieb Scott Lewis:
 
 
 

 On 7/27/2018 7:22 AM, Peter Hermsdorf wrote:
 
 
 
 
 Hi Scott,
 
 
 
 that sounds and looks really promising! I'll need some time to process the information you gave and look at the code and then come back to you.
 
 
 
 
 
 
 This morning I've added some code to allow properties to set the hostname and port on server side, and to use the ecf.endpoint.id on client side to connect to the given hostname and port. This is working...see TCPSocketServerContainer and TCPSocketClientContainer classes.
 
 
 
 I've been testing localhost with the LAN-based discovery (zeroconf) so the hostname/port exported in the endpoint description is what is used on the client. If/when other discovery and import is used on the client, then it can do whatever it wants wrt what hostname and port to use to connect to the server.
 
 
 
 There are some ways that this is limited, but it can be generalized or extended, and I'm hoping it will give a good idea of how to create others.
 
 
 
 Scott
 

 
 
 
   
   
   
   
 

_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Peter Hermsdorf-2

Hi Scott,

sorry about the slow reply. :) 
 

Of course you were right with the value for service.exported.configs. After that the server started successfully.

After fiddling around with some endpoint properties the client could successfully connect to the server and the remote service was imported and could be used.

One issue still exists. After client launch the following exception happens (on client side):

RemoteReferenceNotFoundException[targetID=URIID [uri=tcp://127.0.0.1:3000], idFilter=[URIID [uri=tcp://127.0.0.1:3000]], interfaces=[service.IHello], rsFilter=null]
    at org.eclipse.ecf.osgi.services.remoteserviceadmin.RemoteServiceAdmin.importService(RemoteServiceAdmin.java:2243)
    at org.eclipse.ecf.osgi.services.remoteserviceadmin.RemoteServiceAdmin.importService(RemoteServiceAdmin.java:436)
    at service.consumer.ServiceImporter.importService(ServiceImporter.java:39)
    at service.consumer.ServiceImporter.activate(ServiceImporter.java:35)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.felix.scr.impl.inject.BaseMethod.invokeMethod(BaseMethod.java:229)
    at org.apache.felix.scr.impl.inject.BaseMethod.access$500(BaseMethod.java:39)
    at org.apache.felix.scr.impl.inject.BaseMethod$Resolved.invoke(BaseMethod.java:650)
    at org.apache.felix.scr.impl.inject.BaseMethod.invoke(BaseMethod.java:506)
    at org.apache.felix.scr.impl.inject.ActivateMethod.invoke(ActivateMethod.java:307)
    at org.apache.felix.scr.impl.inject.ActivateMethod.invoke(ActivateMethod.java:299)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:298)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:109)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:906)
    at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:879)
    at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:749)
    at org.apache.felix.scr.impl.manager.AbstractComponentManager.enableInternal(AbstractComponentManager.java:675)
    at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:430)
    at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:657)
    at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:341)
    at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:390)
    at org.apache.felix.scr.impl.Activator.access$200(Activator.java:54)
    at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:265)
    at org.apache.felix.utils.extender.AbstractExtender.createExtension(AbstractExtender.java:254)
    at org.apache.felix.utils.extender.AbstractExtender.modifiedBundle(AbstractExtender.java:227)
    at org.apache.felix.utils.extender.AbstractExtender.addingBundle(AbstractExtender.java:187)
    at org.osgi.util.tracker.BundleTracker$Tracked.customizerAdding(BundleTracker.java:469)
    at org.osgi.util.tracker.BundleTracker$Tracked.customizerAdding(BundleTracker.java:1)
    at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)
    at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:229)
    at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:444)
    at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:908)
    at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at org.eclipse.osgi.internal.framework.EquinoxEventPublisher.publishBundleEventPrivileged(EquinoxEventPublisher.java:213)
    at org.eclipse.osgi.internal.framework.EquinoxEventPublisher.publishBundleEvent(EquinoxEventPublisher.java:120)
    at org.eclipse.osgi.internal.framework.EquinoxEventPublisher.publishBundleEvent(EquinoxEventPublisher.java:112)
    at org.eclipse.osgi.internal.framework.EquinoxContainerAdaptor.publishModuleEvent(EquinoxContainerAdaptor.java:168)
    at org.eclipse.osgi.container.Module.publishEvent(Module.java:476)
    at org.eclipse.osgi.container.Module.doStart(Module.java:578)
    at org.eclipse.osgi.container.Module.start(Module.java:449)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1634)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1614)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.doContainerStartLevel(ModuleContainer.java:1585)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1528)
    at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1)
    at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:340)

!ENTRY org.eclipse.ecf.osgi.services.remoteserviceadmin 4 0 2018-08-14 16:03:25.913
!MESSAGE org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.osgi.services.remoteserviceadmin;code=4;message=org.eclipse.ecf.osgi.services.remoteserviceadmin.TopologyManagerImpl:handleRemoteAdminEvent.IMPORT_ERROR:Import error with event=RemoteServiceAdminEvent[containerID=null, getType()=5, getSource()=org.eclipse.ecf.osgi.services.remoteserviceadmin_4.6.800.v20180518-0149 [21], getException()=RemoteReferenceNotFoundException[targetID=URIID [uri=tcp://127.0.0.1:3000], idFilter=[URIID [uri=tcp://127.0.0.1:3000]], interfaces=[service.IHello], rsFilter=null], getImportReference()=null, getExportReference()=null];severity4;exception=null;children=[]]
ImportRegistration[importReference=ImportReference[importEndpoint=null;exception=RemoteReferenceNotFoundException[targetID=URIID [uri=tcp://127.0.0.1:3000], idFilter=[URIID [uri=tcp://127.0.0.1:3000]], interfaces=[service.IHello], rsFilter=null]];closed=false]

Some seconds later however the service binding happens nevertheless and the remote call is successful. Maybe a timing or startup issue....

Till now I was not able to test this setup in our environment, but from what I've seen it should work for us and solve the recent issues regarding service matching with hostnames.

How do you see the project tcpsocket provider regarding further development / maintenance?

Do you see the project as a proof of concept or do you think it is useful enough for a broader user base and continue development?

What open points / issues do you see with the current code base? What needs be be fixed / changed / added from your point of view?


Thanks for your information!

Bye Peter


Am 09.08.2018 um 01:26 schrieb Scott Lewis:
Hi Peter

Sorry about the slow reply.  I'm out on a gig.

I think the service.exported.configs prop value should be

ecf.socket.server

As per
https://github.com/ECF/tcpsocketdistributionprovider/blob/master/bundles/org.eclipse.ecf.provider.tcpsocket.common/src/org/eclipse/ecf/provider/tcpsocket/common/TCPSocketConstants.java

RE the genetic provider...it was done long before rsa...and there is the peer services also as discussed.

Hope this helps.

Scott 

From: "Peter Hermsdorf" &lt;[hidden email]&gt;
Date: Tue, Aug 7, 2018 7:07 am
To: [hidden email]
Subject: Re: [ecf-dev] Problem with Service Endpoint matching when using different network names
Hi Scott,
  thanks again for providing this sample implementation! That makes it very transparent how things work. 

 Actually I thought the ecf-generic provider would work that way ;)

 

 I wasn't able to get the server up and running. Maybe a problem with the launch configuration. See below for the bundles and their states.

 I added a sample service which should by exported by adding something like

 

 @Component(property = {

   "service.exported.interfaces=*",

   "service.exported.configs=ecf.namespace.socket"

   })

 public class HelloService implements IHello{

 

  public HelloService() {

   System.err.println();

  }

  

  @Override

  public void sayHello() {

   System.err.println("Hello");

  }

 

 }

 

 but as said before, the server does not seem to get started. The start code in the Activator does the registration but the RemoteServiceContainerInstantiator.createInstance is never called. 

 Any idea?

 

 Thanks!

 

 Bye Peter

 

 ss

 

 id State Bundle

 0 ACTIVE org.eclipse.osgi_3.12.50.v20170928-1321

   Fragments=22

 2 ACTIVE org.eclipse.equinox.common_3.9.0.v20170207-1454

 5 ACTIVE org.eclipse.ecf.remoteservice.asyncproxy_2.1.0.v20180409-2248

 6 ACTIVE javax.inject_1.0.0.v20091030

 8 ACTIVE org.eclipse.equinox.event_1.4.0.v20170105-1446

 9 ACTIVE javax.annotation_1.2.0.v201602091430

 11 ACTIVE org.eclipse.osgi.util_3.4.0.v20170111-1608

 14 ACTIVE com.ibm.icu_58.2.0.v20170418-1837

 18 ACTIVE org.eclipse.equinox.concurrent_1.1.0.v20130327-1442

 19 ACTIVE org.eclipse.osgi.services_3.6.0.v20170228-1906

 22 RESOLVED org.eclipse.osgi.compatibility.state_1.1.0.v20170516-1513

   Master=0

 23 ACTIVE org.eclipse.equinox.preferences_3.7.0.v20170126-2132

 25 ACTIVE org.apache.batik.css_1.8.0.v20170214-1941

 26 ACTIVE org.eclipse.core.contenttype_3.6.0.v20170207-1037

 27 ACTIVE org.eclipse.core.jobs_3.9.2.v20171030-1027

 31 ACTIVE org.apache.felix.scr_2.0.10.v20170501-2007

 33 ACTIVE org.eclipse.ecf.identity_3.9.0.v20180426-1936

 34 ACTIVE org.eclipse.equinox.registry_3.7.0.v20170222-1344

 41 ACTIVE org.w3c.css.sac_1.3.1.v200903091627

 43 ACTIVE org.eclipse.ecf.provider.tcpsocket.common_1.0.0.qualifier

 48 ACTIVE org.apache.commons.jxpath_1.3.0.v200911051830

 50 ACTIVE org.eclipse.ecf_3.9.0.v20180402-2015

   Fragments=52

 51 ACTIVE org.apache.batik.util_1.8.0.v20170214-1941

 52 RESOLVED org.eclipse.ecf.ssl_1.2.100.v20180301-0132

   Master=50

 58 ACTIVE org.eclipse.core.runtime_3.13.0.v20170207-1030

 68 ACTIVE org.eclipse.ecf.provider.tcpsocket.server_1.0.0.qualifier

 70 ACTIVE org.eclipse.ecf.remoteservice_8.13.0.v20180406-1915

 73 ACTIVE org.eclipse.equinox.app_1.3.400.v20150715-1528

 75 ACTIVE org.eclipse.ecf.provider.remoteservice_4.4.100.v20180516-2213

 76 ACTIVE org.eclipse.ecf.provider_4.8.0.v20180402-2103

 77 ACTIVE org.eclipse.ecf.sharedobject_2.6.0.v20180404-2345

 78 ACTIVE org.apache.felix.gogo.runtime_0.10.0.v201209301036

 79 ACTIVE org.apache.felix.gogo.shell_0.10.0.v201212101605

 80 ACTIVE org.eclipse.equinox.console_1.1.300.v20170512-2111

 81 ACTIVE service.api_1.0.0.qualifier

 82 ACTIVE service.provider_1.0.0.qualifier

 83 ACTIVE org.eclipse.ecf.discovery_5.0.300.v20180306-0211

 84 ACTIVE org.eclipse.ecf.osgi.services.distribution_2.1.200.v20180301-0016

 85 ACTIVE org.eclipse.ecf.osgi.services.remoteserviceadmin.proxy_1.0.100.v20180301-0016

 86 ACTIVE org.eclipse.ecf.osgi.services.remoteserviceadmin_4.6.800.v20180518-0149

 87 ACTIVE org.eclipse.ecf.provider.jmdns_4.3.200.v20180306-0429

 88 ACTIVE org.eclipse.osgi.services.remoteserviceadmin_1.6.200.v20180301-0016

 89 ACTIVE org.eclipse.ecf.remoteservice.eventadmin_1.3.0.v20180426-2032

 90 ACTIVE org.eclipse.ecf.server_2.1.200.v20180302-2006

 91 ACTIVE org.eclipse.equinox.ds_1.5.0.v20170307-1429

 

 

 

 Am 27.07.2018 um 20:08 schrieb Scott Lewis:
 
 
 

 On 7/27/2018 7:22 AM, Peter Hermsdorf wrote: 
 
 
  
  
 Hi Scott, 
  
 
  
 that sounds and looks really promising! I'll need some time to process the information you gave and look at the code and then come back to you. 
  
 
  
 
  
 
 This morning I've added some code to allow properties to set the hostname and port on server side, and to use the ecf.endpoint.id on client side to connect to the given hostname and port. This is working...see TCPSocketServerContainer and TCPSocketClientContainer classes. 
 
 
 
 I've been testing localhost with the LAN-based discovery (zeroconf) so the hostname/port exported in the endpoint description is what is used on the client. If/when other discovery and import is used on the client, then it can do whatever it wants wrt what hostname and port to use to connect to the server. 
 
 
 
 There are some ways that this is limited, but it can be generalized or extended, and I'm hoping it will give a good idea of how to create others. 
 
 
 
 Scott 
 

_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Scott Lewis-2
Hi Peter,

On 8/14/2018 7:33 AM, Peter Hermsdorf wrote:

Hi Scott,

sorry about the slow reply. :) 
 

Of course you were right with the value for service.exported.configs. After that the server started successfully.

After fiddling around with some endpoint properties the client could successfully connect to the server and the remote service was imported and could be used.

One issue still exists. After client launch the following exception happens (on client side):

<stack deleted>

Some seconds later however the service binding happens nevertheless and the remote call is successful. Maybe a timing or startup issue..


Hmm.   Haven't seen that in my own tests with the example services.   What app environments are your impl and consumer running in? (e.g. RCP, equinox server, etc).  Do you have start levels and if so what are things set at?   I would appreciate if you would open an issue for this here:

https://github.com/ECF/tcpsocketdistributionprovider/issues

..

Till now I was not able to test this setup in our environment, but from what I've seen it should work for us and solve the recent issues regarding service matching with hostnames.

How do you see the project tcpsocket provider regarding further development / maintenance?


That's a good question.    It's reasonably complete now...as it allows multiple services to be exported from a single server socket.   This functionality (multiple services) needs to be tested, however.

One enhancement that might be useful would be to separate out the serialization so that the current object serialization could be replaced with something else (e.g. protobuf) without creating a brand new provider.

Another possible enhancement would be adding and testing sslsocket support.

Yet another would be separating out some sort of authentication.

And it could/will require more testing with bug fixes.

I've got the maven meta-data in to automatically build on travis...it would be nice if it could get built and deployed to maven central eventually.   In short term I would like to produce a github release for 1.0.0.

Do you see the project as a proof of concept or do you think it is useful enough for a broader user base and continue development?


I think it is useful enough to be used as a distribution provider and I would like to continue development.    I also would like a tutorial based upon it so that others could use.

The sticking point for me is that my plate is very full this fall...which is one reason I did the work on it when I did.

If you would like and am able wrt employer, I would welcome contributions (enhancements, issues and fixes, samples, tutorials, docs, etc) from you.   I could make you a team member to do so.

What open points / issues do you see with the current code base? What needs be be fixed / changed / added from your point of view?


Other than the timing issues you discovered, I'm not aware of any obvious bugs but it hasn't been tested that much.  

The things I listed above probably will have to be added eventually, if it's used by multiple folks for more than running the samples. 
 
Thanks,

Scott



Thanks for your information!

Bye Peter


Am 09.08.2018 um 01:26 schrieb Scott Lewis:
Hi Peter

Sorry about the slow reply.  I'm out on a gig.

I think the service.exported.configs prop value should be

ecf.socket.server

As per
https://github.com/ECF/tcpsocketdistributionprovider/blob/master/bundles/org.eclipse.ecf.provider.tcpsocket.common/src/org/eclipse/ecf/provider/tcpsocket/common/TCPSocketConstants.java

RE the genetic provider...it was done long before rsa...and there is the peer services also as discussed.

Hope this helps.

Scott 

From: "Peter Hermsdorf" &lt;[hidden email]&gt;
Date: Tue, Aug 7, 2018 7:07 am
To: [hidden email]
Subject: Re: [ecf-dev] Problem with Service Endpoint matching when using different network names
Hi Scott,
  thanks again for providing this sample implementation! That makes it very transparent how things work. 

 Actually I thought the ecf-generic provider would work that way ;)

 

 I wasn't able to get the server up and running. Maybe a problem with the launch configuration. See below for the bundles and their states.

 I added a sample service which should by exported by adding something like

 

 @Component(property = {

   "service.exported.interfaces=*",

   "service.exported.configs=ecf.namespace.socket"

   })

 public class HelloService implements IHello{

 

  public HelloService() {

   System.err.println();

  }

  

  @Override

  public void sayHello() {

   System.err.println("Hello");

  }

 

 }

 

 but as said before, the server does not seem to get started. The start code in the Activator does the registration but the RemoteServiceContainerInstantiator.createInstance is never called. 

 Any idea?

 

 Thanks!

 

 Bye Peter

 

 ss

 

 id State Bundle

 0 ACTIVE org.eclipse.osgi_3.12.50.v20170928-1321

   Fragments=22

 2 ACTIVE org.eclipse.equinox.common_3.9.0.v20170207-1454

 5 ACTIVE org.eclipse.ecf.remoteservice.asyncproxy_2.1.0.v20180409-2248

 6 ACTIVE javax.inject_1.0.0.v20091030

 8 ACTIVE org.eclipse.equinox.event_1.4.0.v20170105-1446

 9 ACTIVE javax.annotation_1.2.0.v201602091430

 11 ACTIVE org.eclipse.osgi.util_3.4.0.v20170111-1608

 14 ACTIVE com.ibm.icu_58.2.0.v20170418-1837

 18 ACTIVE org.eclipse.equinox.concurrent_1.1.0.v20130327-1442

 19 ACTIVE org.eclipse.osgi.services_3.6.0.v20170228-1906

 22 RESOLVED org.eclipse.osgi.compatibility.state_1.1.0.v20170516-1513

   Master=0

 23 ACTIVE org.eclipse.equinox.preferences_3.7.0.v20170126-2132

 25 ACTIVE org.apache.batik.css_1.8.0.v20170214-1941

 26 ACTIVE org.eclipse.core.contenttype_3.6.0.v20170207-1037

 27 ACTIVE org.eclipse.core.jobs_3.9.2.v20171030-1027

 31 ACTIVE org.apache.felix.scr_2.0.10.v20170501-2007

 33 ACTIVE org.eclipse.ecf.identity_3.9.0.v20180426-1936

 34 ACTIVE org.eclipse.equinox.registry_3.7.0.v20170222-1344

 41 ACTIVE org.w3c.css.sac_1.3.1.v200903091627

 43 ACTIVE org.eclipse.ecf.provider.tcpsocket.common_1.0.0.qualifier

 48 ACTIVE org.apache.commons.jxpath_1.3.0.v200911051830

 50 ACTIVE org.eclipse.ecf_3.9.0.v20180402-2015

   Fragments=52

 51 ACTIVE org.apache.batik.util_1.8.0.v20170214-1941

 52 RESOLVED org.eclipse.ecf.ssl_1.2.100.v20180301-0132

   Master=50

 58 ACTIVE org.eclipse.core.runtime_3.13.0.v20170207-1030

 68 ACTIVE org.eclipse.ecf.provider.tcpsocket.server_1.0.0.qualifier

 70 ACTIVE org.eclipse.ecf.remoteservice_8.13.0.v20180406-1915

 73 ACTIVE org.eclipse.equinox.app_1.3.400.v20150715-1528

 75 ACTIVE org.eclipse.ecf.provider.remoteservice_4.4.100.v20180516-2213

 76 ACTIVE org.eclipse.ecf.provider_4.8.0.v20180402-2103

 77 ACTIVE org.eclipse.ecf.sharedobject_2.6.0.v20180404-2345

 78 ACTIVE org.apache.felix.gogo.runtime_0.10.0.v201209301036

 79 ACTIVE org.apache.felix.gogo.shell_0.10.0.v201212101605

 80 ACTIVE org.eclipse.equinox.console_1.1.300.v20170512-2111

 81 ACTIVE service.api_1.0.0.qualifier

 82 ACTIVE service.provider_1.0.0.qualifier

 83 ACTIVE org.eclipse.ecf.discovery_5.0.300.v20180306-0211

 84 ACTIVE org.eclipse.ecf.osgi.services.distribution_2.1.200.v20180301-0016

 85 ACTIVE org.eclipse.ecf.osgi.services.remoteserviceadmin.proxy_1.0.100.v20180301-0016

 86 ACTIVE org.eclipse.ecf.osgi.services.remoteserviceadmin_4.6.800.v20180518-0149

 87 ACTIVE org.eclipse.ecf.provider.jmdns_4.3.200.v20180306-0429

 88 ACTIVE org.eclipse.osgi.services.remoteserviceadmin_1.6.200.v20180301-0016

 89 ACTIVE org.eclipse.ecf.remoteservice.eventadmin_1.3.0.v20180426-2032

 90 ACTIVE org.eclipse.ecf.server_2.1.200.v20180302-2006

 91 ACTIVE org.eclipse.equinox.ds_1.5.0.v20170307-1429

 

 

 

 Am 27.07.2018 um 20:08 schrieb Scott Lewis:
 
 
 

 On 7/27/2018 7:22 AM, Peter Hermsdorf wrote: 
 
 
  
  
 Hi Scott, 
  
 
  
 that sounds and looks really promising! I'll need some time to process the information you gave and look at the code and then come back to you. 
  
 
  
 
  
 
 This morning I've added some code to allow properties to set the hostname and port on server side, and to use the ecf.endpoint.id on client side to connect to the given hostname and port. This is working...see TCPSocketServerContainer and TCPSocketClientContainer classes. 
 
 
 
 I've been testing localhost with the LAN-based discovery (zeroconf) so the hostname/port exported in the endpoint description is what is used on the client. If/when other discovery and import is used on the client, then it can do whatever it wants wrt what hostname and port to use to connect to the server. 
 
 
 
 There are some ways that this is limited, but it can be generalized or extended, and I'm hoping it will give a good idea of how to create others. 
 
 
 
 Scott 
 


_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev



_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Scott Lewis-2
On 8/14/2018 8:11 AM, Scott Lewis wrote:
> <stuff deleted>

> That's a good question.    It's reasonably complete now...as it allows
> multiple services to be exported from a single server socket.   This
> functionality (multiple services) needs to be tested, however.
>
> One enhancement that might be useful would be to separate out the
> serialization so that the current object serialization could be
> replaced with something else (e.g. protobuf) without creating a brand
> new provider.
>
> Another possible enhancement would be adding and testing sslsocket
> support.
>
> Yet another would be separating out some sort of authentication.
>
> And it could/will require more testing with bug fixes.
>
> I've got the maven meta-data in to automatically build on travis...it
> would be nice if it could get built and deployed to maven central
> eventually.   In short term I would like to produce a github release
> for 1.0.0.

Also:  Use of websockets.

Scott


_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Service Endpoint matching when using different network names

Peter Hermsdorf-2
In reply to this post by Scott Lewis-2

Hi,

sorry for the slow reply.


I think it is useful enough to be used as a distribution provider and I would like to continue development.    I also would like a tutorial based upon it so that others could use.

The sticking point for me is that my plate is very full this fall...which is one reason I did the work on it when I did.

If you would like and am able wrt employer, I would welcome contributions (enhancements, issues and fixes, samples, tutorials, docs, etc) from you.   I could make you a team member to do so.
I would like to do so. That's also fine with my employer.
The point is, that it will not happen immediately as I currently have other important topics to work on.

But it's a good and transparent implementation which serves our use case very well, so I'll come back on that some time later.
Currently we have some workarounds (DNS/host entries) which allow us to gain some time on this topic...

First thing for me would probably be to implement it in our system and see how it works out.

Thanks again!

Bye Peter


_______________________________________________
ecf-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ecf-dev