• Going from java monolith to microservice

    From mike@21:1/5 to All on Sat Jan 16 06:29:48 2021
    Hi,

    First I want to say that this is not a pure "how do I do this in java" - question.
    So if you are not interested or don't have any experience with the topic feel free to skip.

    We have a library that consists of different maven modules. It is basically different abstractions to test NETCONF devices.

    We hear a lot of positives using microservices since it will be easier to deploy separate and independen. So I thought that it might be possible to transition over to use microservices since products tends to be virtualized in the cloud.

    Do you have any idea on how to do this journey? My guess is to make a step to step transition. We need to support the old way of using the monolith for our old products.

    My first thought will be how to use the service? Today testers write java code but what will the testing look like in the future? Have anyone gone from using java to microsevice. What will be the main difference. I know my questions are vague and that is
    where you come in. I need guidance and advice in this matter. I would really appreciate some advice.

    br,

    //mike

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From mike@21:1/5 to All on Sat Jan 16 08:44:18 2021
    lördag 16 januari 2021 kl. 17:17:00 UTC+1 skrev Graeme Geldenhuys:
    On 16/01/2021 2:29 pm, mike wrote:
    We hear a lot of positives using microservices since it will
    be easier to deploy separate and independen. So I thought that
    it might be possible to transition over to use microservices since products tends to be virtualized in the cloud.
    Our company transitioned from a huge monolith to 35+ Microservices. Microservices bring with them their own set of problems,
    so they are definitely not the "silver bullet" to fixing the problems associated with a monolith project.

    I would highly recommend you do some internet searches on reasons not
    to use a Microservice - just so you get the facts from both sides
    and have more information before you start making changes to your code.

    The question you should ask is:

    What do you think the Microservice Architecture will give you?

    Yes I agree. It is a good question. I am not sure that is why I want to investigate.

    A monolith project doesn't need to be maintenance nightmare either. It
    can be cleaned up and refactored.

    I agree. Today we have a set of 6 maven modules. There are some dependencies between them.
    Since the usage is from low level up to high level of programming for testers to use.
    I guess a good start is to refactor code and see how much we really need and what could be in separate new modules.

    The javadoc we have for users to use is not as good as I wanted since it is for all public classes. Even those that is not
    intented for external usage. That is becasue we are currently stuck with java 8. We plan for java 12 but it takes time.
    So it does not become clear to testers what the should use.


    A possible alternative is to embrace Java 11 and split your project up using Java Modules to represent domains in your business model. You could have many of the same benefits Microservices offer and also new improvements, like much improved latency and memory usage, easier deployments, easier security model etc.


    I see a possible benefint for testing. Where we can use https://www.testcontainers.org/ for a number of open source devices
    to run unit test with.
    Many of our Microservices became a nightmare to maintain too, so we
    started introducing the Ports & Adapters architecture inside a Microservice project to help improve things. This could equally have been applied
    to a Monolith project too.

    //mike

    Regards,
    Graeme

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Graeme Geldenhuys@21:1/5 to mike on Sat Jan 16 16:16:46 2021
    On 16/01/2021 2:29 pm, mike wrote:
    We hear a lot of positives using microservices since it will
    be easier to deploy separate and independen. So I thought that
    it might be possible to transition over to use microservices since
    products tends to be virtualized in the cloud.

    Our company transitioned from a huge monolith to 35+ Microservices. Microservices bring with them their own set of problems,
    so they are definitely not the "silver bullet" to fixing the problems associated with a monolith project.

    I would highly recommend you do some internet searches on reasons not
    to use a Microservice - just so you get the facts from both sides
    and have more information before you start making changes to your code.

    The question you should ask is:

    What do you think the Microservice Architecture will give you?

    A monolith project doesn't need to be maintenance nightmare either. It
    can be cleaned up and refactored.

    A possible alternative is to embrace Java 11 and split your project up using Java Modules to represent domains in your business model. You could have
    many of the same benefits Microservices offer and also new improvements,
    like much improved latency and memory usage, easier deployments, easier security model etc.

    Many of our Microservices became a nightmare to maintain too, so we
    started introducing the Ports & Adapters architecture inside a Microservice project to help improve things. This could equally have been applied
    to a Monolith project too.

    Regards,
    Graeme

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Arne_Vajh=c3=b8j?=@21:1/5 to mike on Sat Jan 16 15:24:08 2021
    On 1/16/2021 9:29 AM, mike wrote:
    First I want to say that this is not a pure "how do I do this in
    java" - question. So if you are not interested or don't have any
    experience with the topic feel free to skip.

    We have a library that consists of different maven modules. It is
    basically different abstractions to test NETCONF devices.

    We hear a lot of positives using microservices since it will be
    easier to deploy separate and independen. So I thought that it might
    be possible to transition over to use microservices since products
    tends to be virtualized in the cloud.

    Do you have any idea on how to do this journey? My guess is to make a
    step to step transition. We need to support the old way of using the
    monolith for our old products.

    My first thought will be how to use the service? Today testers write
    java code but what will the testing look like in the future? Have
    anyone gone from using java to microsevice. What will be the main
    difference. I know my questions are vague and that is where you come
    in. I need guidance and advice in this matter. I would really
    appreciate some advice.

    Micro-services vs less granular services vs monolith is a well-known
    discussion - you should be able to find hundreds of articles and books
    about that topic from a general perspective.

    But I am somewhat puzzled by the talk about library/modules and
    test.

    In its core micro-services is about the runtime separation of
    services.

    Let us say that before is:

    Tomcat port 8080
    a.war with libx.jar and liby.jar, exposing http://xxxxxx:8080/a/foo
    b.war with libx.jar and libz.jar, exposing http://xxxxxx:8080/b/bar

    And after is:

    Spring Boot a.jar with libx.jar and liby.jar, exposing
    http://xxxxxx:8081/a/foo
    Spring Boot b.jar with libx.jar and libz.jar, exposing
    http://xxxxxx:8082/b/foo

    You have not changed the libraries.

    And I do not see any reason why you would change test either.

    You have gotten some benefits. The services can be scaled independently
    (you can run 2 instances of a and 10 instances of b). The services do
    no longer share the global Tomcat context (database connection
    pools etc.).

    If before is:

    Tomcat port 8080
    kitchensink.war with libx.jar and liby.jar and libz.jar, exposing http://xxxxxx:8080/kitchensink/foo and http://xxxxxx:8080/kitchensink/bar

    then splitting up in micro-services will force a separation of code
    and dependencies.

    And that may impact a lot more.

    But that is not really a micro-service thing - you would have to do the
    same if you split kitchensink.war in a.war and b.war.

    Arne

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From mike@21:1/5 to All on Sun Jan 17 05:08:12 2021
    lördag 16 januari 2021 kl. 21:24:31 UTC+1 skrev Arne Vajhøj:
    On 1/16/2021 9:29 AM, mike wrote:
    First I want to say that this is not a pure "how do I do this in
    java" - question. So if you are not interested or don't have any experience with the topic feel free to skip.

    We have a library that consists of different maven modules. It is basically different abstractions to test NETCONF devices.

    We hear a lot of positives using microservices since it will be
    easier to deploy separate and independen. So I thought that it might
    be possible to transition over to use microservices since products
    tends to be virtualized in the cloud.

    Do you have any idea on how to do this journey? My guess is to make a
    step to step transition. We need to support the old way of using the monolith for our old products.

    My first thought will be how to use the service? Today testers write
    java code but what will the testing look like in the future? Have
    anyone gone from using java to microsevice. What will be the main difference. I know my questions are vague and that is where you come
    in. I need guidance and advice in this matter. I would really
    appreciate some advice.
    Micro-services vs less granular services vs monolith is a well-known discussion - you should be able to find hundreds of articles and books
    about that topic from a general perspective.

    But I am somewhat puzzled by the talk about library/modules and
    test.

    In its core micro-services is about the runtime separation of
    services.

    Let us say that before is:

    Tomcat port 8080
    a.war with libx.jar and liby.jar, exposing http://xxxxxx:8080/a/foo
    b.war with libx.jar and libz.jar, exposing http://xxxxxx:8080/b/bar

    And after is:

    Spring Boot a.jar with libx.jar and liby.jar, exposing http://xxxxxx:8081/a/foo
    Spring Boot b.jar with libx.jar and libz.jar, exposing http://xxxxxx:8082/b/foo

    You have not changed the libraries.

    Hi Arne,

    The different modules are normally delivered as mononlith ( xzy.jar) since it is more convenient.
    Some can work standalone ( like the lowest layer).
    Then users ( intendend category is NETCONF device testers) can access what ever layers the like:

    High level:

    NetconfClient client = NetconfClientBuilder.setConnectionTimeout(10).usePrettyPrint(true).build();
    client.create();

    Low level:

    NetconfHandler netconf = NetconfHandler.createSshTransport(); netconf.connection(String host,int port);
    netconf.editConfig(Sting config,String target);
    netconf.disconnect();

    So as you can see there is no webservice used today. I guess I was not clear in my first description. Is an API that testers can use create their own test code for the NETCONF enabled device.

    The problem we have today it that there are many public classes that should not actually be used by testers. However we cannot change access since other dependent modules need these classes.

    I thought we could hide them using a microservice and also make it more clear for testers what can be accessed. I realize testers can not write code then ( as above shows) but instead create urls with parameters.

    So,after discussing with you guys, I realize that my real problem is to create a public API for each module
    that is clear and consise for users to work with at any level, approx 5 levels. Top level has dependencies to lower levels ( chained).

    High level
    ......
    Low level

    I guess a microservice will not solve this. Maybe I should not even think microservices. Is there a nother way to solve my problem. Can I do some preparation work in java 8 to move over to java 12 when it is time?

    br.

    //mike

    And I do not see any reason why you would change test either.

    You have gotten some benefits. The services can be scaled independently
    (you can run 2 instances of a and 10 instances of b). The services do
    no longer share the global Tomcat context (database connection
    pools etc.).

    If before is:

    Tomcat port 8080
    kitchensink.war with libx.jar and liby.jar and libz.jar, exposing http://xxxxxx:8080/kitchensink/foo and http://xxxxxx:8080/kitchensink/bar

    then splitting up in micro-services will force a separation of code
    and dependencies.

    And that may impact a lot more.

    But that is not really a micro-service thing - you would have to do the
    same if you split kitchensink.war in a.war and b.war.

    Arne

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Graeme Geldenhuys@21:1/5 to mike on Sun Jan 17 16:32:43 2021
    On 17/01/2021 1:08 pm, mike wrote:
    The problem we have today it that there are many public classes
    that should not actually be used by testers. However we cannot
    change access since other dependent modules need these classes.

    Yes, that is often a trap many teams fall into. They open their classes
    way to much, and then later struggle to introduce tighter control.

    I mentioned it in my initial reply, and will mention it again. I would recommend you read some articles on the Ports and Adapters architecture,
    which can be applied to existing code. We are three-quarters of the
    way in converting an existing project to it, and the results so far is
    a much cleaner design. So I would highly recommend it. It also means
    our Primary Adapters (interfaces into our domain) can be implemented
    using Test code, HTTP request (web service) etc. It becomes very
    flexible on how it can be used.

    Here is one article, but there are many more:

    https://herbertograca.com/2017/09/14/ports-adapters-architecture/

    There are some excellent YouTube videos talking about the subject
    as well.

    Regards,
    Graeme

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Arne_Vajh=c3=b8j?=@21:1/5 to mike on Thu Jan 21 19:02:07 2021
    On 1/17/2021 8:08 AM, mike wrote:
    The different modules are normally delivered as mononlith ( xzy.jar) since it is more convenient.
    Some can work standalone ( like the lowest layer).
    Then users ( intendend category is NETCONF device testers) can access what ever layers the like:

    High level:

    NetconfClient client = NetconfClientBuilder.setConnectionTimeout(10).usePrettyPrint(true).build();
    client.create();

    Low level:

    NetconfHandler netconf = NetconfHandler.createSshTransport(); netconf.connection(String host,int port);
    netconf.editConfig(Sting config,String target);
    netconf.disconnect();

    So as you can see there is no webservice used today. I guess I was not clear in my first description. Is an API that testers can use create their own test code for the NETCONF enabled device.

    The problem we have today it that there are many public classes that should not actually be used by testers. However we cannot change access since other dependent modules need these classes.

    I thought we could hide them using a microservice and also make it more clear for testers what can be accessed. I realize testers can not write code then ( as above shows) but instead create urls with parameters.

    So,after discussing with you guys, I realize that my real problem is to create a public API for each module
    that is clear and consise for users to work with at any level, approx 5 levels. Top level has dependencies to lower levels ( chained).

    High level
    ......
    Low level

    I guess a microservice will not solve this. Maybe I should not even think microservices. Is there a nother way to solve my problem. Can I do some preparation work in java 8 to move over to java 12 when it is time?

    Yep.

    It sounds like you need the new Java module system not micro-services.

    The new Java module system is a pretty good tool for hiding
    "internal public" classes.

    Arne

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)