Skip to content

[BUG] VertexAI Breaks when relying on ADC from Cloud Run #536

@JakeDluhy

Description

@JakeDluhy

Basic checks

  • I searched existing issues - this hasn't been reported
  • I can reproduce this consistently
  • This is a RubyLLM bug, not my application code

What's broken?

This is specifically when using VertexAI.

I have been using ADC successfully both locally and deployed onto heroku with an ADC JSON file. However, I have recently investigated moving my deployments to GCP, and am now deploying onto Cloud Run. As part of this I decided to let the Service Agent that is attached to my cloud run instance handle the ADC through RubyLLM for VertexAI. However specifically on Cloud Run I've been getting an error in initialize_authorizer (see attached stack trace).
stack-trace.txt

I used AI to develop a monkey patch that seems to be working:

# Workaround for ruby_llm 1.9.1 calling:
#   Google::Auth.get_application_default(scope: [...])
#
# In Ruby 3+, that keyword arg becomes a positional Hash for googleauth's
# signature (`get_application_default scope=nil, options={}`), which leads to:
#   TypeError: Expected Array or String, got Hash
#
# Remove once ruby_llm fixes upstream to pass scopes as the first positional arg.
module RubyLLM
  module Providers
    class VertexAI
      private

      def initialize_authorizer
        require "googleauth"

        scopes = [
          "https://www.googleapis.com/auth/cloud-platform",
          "https://www.googleapis.com/auth/generative-language.retriever"
        ]

        @authorizer = ::Google::Auth.get_application_default(scopes)
      rescue LoadError
        raise Error,
              'The googleauth gem ~> 1.15 is required for Vertex AI. Please add it to your Gemfile: gem "googleauth"'
      end
    end
  end
end

Let me know if you need anything else.

How to reproduce

  1. Set up RubyLLM to use VertexAI
  2. Deploy onto Google's Cloud Run with an attached service account that can call VertexAI. Do not set anything else up for application default credentials
  3. Call VertexAI through RubyLLM

Expected behavior

It completes VertexAI call successfully

What actually happened

It errors,

stack-trace.txt

Environment

Ruby Version: 3.4.2
Ruby LLM: 1.9.1
Provider: VertexAI
OS: Linux/Cloud Run

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions