One of the tensions here is latency though -- the more middleware one introduces, the more you slow down API responses for models that are already often borderline intolerably slow. Waiting even 140 something milliseconds (https://thefastest.ai/) for a response is still unacceptably slow for many high volume technical applications.
One of the tensions here is latency though -- the more middleware one introduces, the more you slow down API responses for models that are already often borderline intolerably slow. Waiting even 140 something milliseconds (https://thefastest.ai/) for a response is still unacceptably slow for many high volume technical applications.