Springboot - Circuit Breaker Pattern - Hystrix

Problem Statement
Software systems make remote calls to software running in different processes, usually on different machines across a network. One of the big differences between in-memory calls and remote calls is that remote calls can fail, or hang without a response until some timeout limit is reached. 

Solution

The circuit breaker pattern is the solution to this problem. The basic idea behind the circuit breaker is very simple. You wrap a protected function call in a circuit breaker object, which monitors for failures. Once the failures reach a certain threshold (default is 10, for thread & semaphore both), the circuit breaker trips, and all further calls to the circuit breaker return with an error or with some alternative service or default message, without the protected call being made at all. This will make sure system is responsive and threads are not waiting for an unresponsive call.

Note: basically, circuit breaker makes the remote call by creating a new thread

             --- if this thread fails due to remote service timeout or any exception then circuit breaker will create new thread after certain time in seconds(says 2 seconds).

            --- lets suppose it again fails and it fails for thread threshold count(default is 10) then circuit breaker will be in open state and fallback method gets executed.

Different States of the Circuit Breaker

The circuit breaker has three distinct states: Closed, Open, and Half-Open:

• Closed – When everything is normal, the circuit breaker remains in the closed state and all calls pass through to the services. When the number of failures exceeds a predetermined threshold the breaker trips, and it goes into the Open state.
• Open – The circuit breaker returns an error for calls without executing the function.
• Half-Open – After a timeout period, the circuit switches to a half-open state to test if the underlying problem still exists. If a single call fails in this half-open state, the breaker is once again tripped. If it succeeds, the circuit breaker resets back to the normal, closed state. 

Spring Cloud Netflix Hystrix – the fault tolerance library. We’ll use the library and implement the Circuit Breaker enterprise pattern.

Hystrix helps to make applications resilient and fault tolerant. It improves overall resilience of the system by isolating the failing services and stopping the cascading effect of failures.

1. Add the maven dependency

<dependency>

    <groupId>org.springframework.cloud</groupId>

    <artifactId>spring-cloud-starter-hystrix</artifactId>

    <version>1.1.5.RELEASE</version>

</dependency>

2. We’re going to create a @Service class first, which will be injected to a @Controller. 

This will be our injectable @Service implementing a @HystrixCommand with an associated fallback method. This fallback has to use the same signature as the ‘original’.

3. BookstoreApplication will be our main application class. The @EnableCircuitBreaker annotation will scan the classpath for any compatible Circuit Breaker implementation.

To use Hystrix explicitly, you have to annotate this class with @EnableHystrix:

Example

You can implement the circuit breaker pattern with Netflix Hystrix. The following code can better explain the solution. 

The below microservice recommends the reading list to the customer:

package hello;

import org.springframework.boot.SpringApplication;

import org.springframework.boot.autoconfigure.SpringBootApplication;

import org.springframework.web.bind.annotation.RestController;

import org.springframework.web.bind.annotation.RequestMapping;

@RestController

@SpringBootApplication

public class BookstoreApplication {

  @RequestMapping(value = "/recommended")

  public String readingList(){

    return "Spring in Action (Manning), Cloud Native Java (O'Reilly), Learning Spring Boot (Packt)";

  }

  public static void main(String[] args) {

    SpringApplication.run(BookstoreApplication.class, args);

  }

}

Client application code which will call the reading list recommendation service:

package hello;

import java.net.URI;

import org.springframework.stereotype.Service;

import org.springframework.web.client.RestTemplate;

import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;

@Service

public class BookService {

 private final RestTemplate restTemplate;

 public BookService(RestTemplate rest) {

  this.restTemplate = rest;

 }

 @HystrixCommand(fallbackMethod = "reliable")

 public String readingList() {

  URI uri = URI.create("http://localhost:8090/recommended");

  return this.restTemplate.getForObject(uri, String.class);

 }

 public String reliable() {

  return "Cloud Native Java (O'Reilly)";

 }

}

In the above code method, the reading list is calling remote microservice API to get the reading list recommendation. Look at line number 19 of the above code, we have provided fallback method "reliable." If the remote API does not respond in time, the method "reliable" will be called and that will serve the request.

In the fallback method, you can return either a default output or even call some other remote or local API to serve the request.

Interview Questions and Answers:

explain execution.isolation.thread.timeoutInMilliseconds?

This property sets the time in milliseconds after which the caller will observe a timeout and walk away from the command execution. Hystrix marks the HystrixCommand as a TIMEOUT, and performs fallback logic. Note that there is configuration for turning off timeouts per-command, if that is desired (see command.timeout.enabled).

Default Value	1000

explain circuitBreaker.requestVolumeThreshold?

it say you don't need to open the circuit breaker for minimum number of request.

for example : if you set its value to 20 then it will allow all 20 request even all gets failed but circuit breaker will not trip open OR fallback method will not be called for 20 request.

Default Value	20

explain circuitBreaker.sleepWindowInMilliseconds?

lets suppose circuit breaker is open and all calls are made to fallback method.

this property value (in milliseconds) instruct circuit breaker to go and check whether api service is up and running. if not then circuit breaker remains open otherwise circuit breaker will be in closed state and all further call will be made to api.

Default Value	5000

explain circuitBreaker.errorThresholdPercentage?

This property sets the error percentage at or above which the circuit should trip open and start short-circuiting requests to fallback logic.

For example: if I set 20% then it means if total 100 calls were made to api and out of 100 if 20 or above fail then circuit should be open and fallback method should be called.

Default Value	50

is @HystrixCommand a synchronous call or not

Yes

can we make @HystrixCommand as a asynchronous call or not?

Yes, using the below code snippet.

To process Hystrix command asynchronously you should return an instance of AsyncResult in your command method as in the example below:

    @HystrixCommand
    public Future<User> getUserByIdAsync(final String id) {
        return new AsyncResult<User>() {
            @Override
            public User invoke() {
                return userResource.getUserById(id);
            }
        };
    }

The return type of command method should be Future that indicates that a command should be executed [asynchronously] (https://github.com/Netflix/Hystrix/wiki/How-To-Use#wiki-Asynchronous-Execution).

Reactive Execution

To perform "Reactive Execution" you should return an instance of Observable in your command method as in the example below:

    @HystrixCommand
    public Observable<User> getUserById(final String id) {
        return Observable.create(new Observable.OnSubscribe<User>() {
                @Override
                public void call(Subscriber<? super User> observer) {
                    try {
                        if (!observer.isUnsubscribed()) {
                            observer.onNext(new User(id, name + id));
                            observer.onCompleted();
                        }
                    } catch (Exception e) {
                        observer.onError(e);
                    }
                }
            });
    }

In addition to Observable Javanica supports the following RX types: Single and Completable. Hystrix core supports only one RX type which is Observable, HystrixObservableCommand requires to return Observable therefore javanica transforms Single or Completable to Observable using toObservable() method for appropriate type and before returning the result to caller it translates Observable to either Single or Completable using toSingle() or toCompletable() correspondingly.

HystrixObservable interface provides two methods: observe() - eagerly starts execution of the command the same as HystrixCommand#queue() and HystrixCommand#execute(); toObservable() - lazily starts execution of the command only once the Observable is subscribed to. To control this behaviour and swith between two modes @HystrixCommand provides specific parameter called observableExecutionMode. @HystrixCommand(observableExecutionMode = EAGER) indicates that observe() method should be used to execute observable command @HystrixCommand(observableExecutionMode = LAZY) indicates that toObservable() should be used to execute observable command

can we ignore/escape fallback_method for any specific exception?

Yes, we can add below property and can specify type of the exceptions

@HystrixCommand(

  fallbackMethod = "findCachedRatingById",

  ignoreExceptions = { RatingNotFoundException.class })

HystrixCommand annotation - what is purpose of commandKey?

I want to know what the commandKey param is. In the following context i want to know what this parameter means:

@HystrixCommand(groupKey="UserGroup", commandKey = "GetUserByIdCommand")

public User getUserById(String id) {

    return userResource.getUserById(id);

}

By default the name of command key is command method name: For example , getUserById but you can rename it to getUserByIdCommand

Then you can use the commandKey in hystrix commands to reference the methods. If you dont use the commandKey (its optional). then the method name is used as default. So its just to rename the command.

Explain Default fallback for class or concrete command

This feature allows to define default fallback for the whole class or concrete command. If you have a batch of commands with exactly the same fallback logic you still have to define a fallback method for every command because fallback method should have exactly the same signature as command does, consider the following code:

    public class Service {
        @RequestMapping(value = "/test1")
        @HystrixCommand(fallbackMethod = "fallback")
        public APIResponse test1(String param1) {
            // some codes here
            return APIResponse.success("success");
        }

        @RequestMapping(value = "/test2")
        @HystrixCommand(fallbackMethod = "fallback")
        public APIResponse test2() {
            // some codes here
            return APIResponse.success("success");
        }

        @RequestMapping(value = "/test3")
        @HystrixCommand(fallbackMethod = "fallback")
        public APIResponse test3(ObjectRequest obj) {
            // some codes here
            return APIResponse.success("success");
        }

        private APIResponse fallback(String param1) {
            return APIResponse.failed("Server is busy");
        }

        private APIResponse fallback() {
            return APIResponse.failed("Server is busy");
        }
        
        private APIResponse fallback(ObjectRequest obj) {
            return APIResponse.failed("Server is busy");
        }
    }

Default fallback feature allows to engage DRY principle and get rid of redundancy:

    @DefaultProperties(defaultFallback = "fallback")
    public class Service {
        @RequestMapping(value = "/test1")
        @HystrixCommand
        public APIResponse test1(String param1) {
            // some codes here
            return APIResponse.success("success");
        }

        @RequestMapping(value = "/test2")
        @HystrixCommand
        public APIResponse test2() {
            // some codes here
            return APIResponse.success("success");
        }

        @RequestMapping(value = "/test3")
        @HystrixCommand
        public APIResponse test3(ObjectRequest obj) {
            // some codes here
            return APIResponse.success("success");
        }

        private APIResponse fallback() {
            return APIResponse.failed("Server is busy");
        }
    }

Default fallback method should not have any parameters except extra one to get execution exception and shouldn't throw any exceptions. Below fallbacks listed in descending order of priority:

1. command fallback defined using fallbackMethod property of @HystrixCommand
2. command default fallback defined using defaultFallback property of @HystrixCommand
3. class default fallback defined using defaultFallback property of @DefaultProperties

What is Error Propagation?

Based on this description, @HystrixCommand has an ability to specify exceptions types which should be ignored.

    @HystrixCommand(ignoreExceptions = {BadRequestException.class})
    public User getUserById(String id) {
        return userResource.getUserById(id);
    }

If userResource.getUserById(id); throws an exception that type is BadRequestException then this exception will be wrapped in HystrixBadRequestException and re-thrown without triggering fallback logic. You don't need to do it manually, javanica will do it for you under the hood.

It is worth noting that by default a caller will always get the root cause exception e.g. BadRequestException, never HystrixBadRequestException or HystrixRuntimeException (except the case when executed code explicitly throws those exceptions).

Optionally this exception un-wrapping can be disabled for HystrixRuntimeException by using raiseHystrixExceptions i.e. all exceptions that are not ignored are raised as the cause of a HystrixRuntimeException:

    @HystrixCommand(
        ignoreExceptions = {BadRequestException.class},
        raiseHystrixExceptions = {HystrixException.RUNTIME_EXCEPTION})
    public User getUserById(String id) {
        return userResource.getUserById(id);
    }

Note: If command has a fallback then only first exception that triggers fallback logic will be propagated to caller. Example:

class Service {
    @HystrixCommand(fallbackMethod = "fallback")
    Object command(Object o) throws CommandException {
        throw new CommandException();
    }
    
    @HystrixCommand
    Object fallback(Object o) throws FallbackException {
        throw new FallbackException();
    }
}

// in client code
{
    try {
        service.command(null);
    } catch (Exception e) {
      assert CommandException.class.equals(e.getClass())
    }
}

https://github.com/Netflix/Hystrix/wiki/Configuration#execution.isolation.thread.timeoutInMilliseconds

SHARE

esant technology

• 
• 
• 
• 
•