gm_auth_configure() and gm_send_message() work locally but not remotely on Cloud Build with same JSON file (or key and secret) specified

[sommerhd-royals]

I did not see this specific issue raised by others so submitting a new issue here.

I am able to use the gmailr library to send emails with no problems locally, using a json file created as a "Desktop App" Application type through the "Create Credentials -> OAuth 2.0 Client IDs" workflow from the Credentials tab on Gmail API.

To confirm, the below works fine locally -
>library(googleCloudStorageR)
>library(gmailr)

SET GCS BUCKET NAME -
> gcs_global_bucket("our_preferred_bucket_name")

Show json file path is correct -
> objects <- gcs_list_objects() # data from the bucket as list

Show JSON file name is correct
> print(objects$name[4])
[1] "file_for_email_sending.json"
> gm_auth_configure(path = "file_for_email_sending.json", email = "my_email_addess.com")

Can create email here -
> test_message <- gm_mime() %>% gm_to("my_email_addess.com") %>% gm_from("my_email_addess.com") %>% gm_subject("blank subject") %>% gm_text_body("blank body")

Email send below works fine locally
> gm_send_message(test_message)

However, the same approach simply will not work for me when I test run an R script using Cloud Build (and googleCloudRunner I should add). When I use the path reference approach shown above, I receive the below error trying to run gm_send_message():

Error: lexical error: invalid char in json text.
file_for_email_sending.j
(right here) ------^

When I try manually specifying the key and secret (client_id and client_secret respectively in the JSON file), I receive the below Gmail API 403 error when I run gm_send_message():

Error in gmailr_POST(c("messages", "send"), user_id, class = "gmail_message", :
Gmail API error: 403
Request had insufficient authentication scopes.
Calls: gm_send_message -> gmailr_POST -> gmailr_query
Execution halted

I would greatly appreciate any pro tips on how to resolve.

[jennybc]

I think the first thing to do is to gather more precise data about the success vs. failure.

Make gargle more verbose as described here:

https://gargle.r-lib.org/articles/troubleshooting.html

It might also be interesting to see the backtrace for "Error: lexical error: invalid char in json text".

You might want to call gm_oauth_app() explicitly to make sure the app is what you think it is.

Probably also interesting to call gm_profile() in both settings.

[jennybc]

We won't know for sure until you do more research, but it's also possible that, in the Cloud Build environment, you are actually authing as the default service account, not yourself. That is what I suspect, but we need more information.

[sommerhd-royals]

Hi @jennybc - Greatly appreciate the tips. I tried them to the best of my abilities. Unfortunately, I'm still stuck on a 403 error when I try to run in Cloud Build. Here's what my error looks like, just FYI to anyone experiencing something similar:

Error in gmailr_POST(c("messages", "send"), user_id, class = "gmail_message",  : 
  Gmail API error: 403
  Request had insufficient authentication scopes.
Calls: gm_send_message -> gmailr_POST -> gmailr_query

And just for reference, I used Damien Caillet's guide at https://medium.com/@damiencaillet/automate-r-code-in-the-cloud-89266910cc36 which could not have been any smoother for me when it comes to porting kind of the basic R workflow of loading R packages, authenticating to Google Cloud Storage, reading from a DB, doing data manipulation, writing out to GCP tables, etc. It's just "email sending capability in the Cloud using Cloud Build" that I'm stuck on. I mention this as I realize gmailr was built before googleCloudRunner and just wanted to double check these two things are considered compatible today.

I have an open ticket with Google Support, and a lot of colleagues in my line of work who both (a) are R users and (b) are new to Google Cloud like me who might be able to help, so if Support doesn't help I'll start asking around, & I'll be sure to post out here what I find so that hopefully no one else goes through my issue again.

I think the issue is that the "App" I created is in a "Needs Verification" state, which for whatever reason doesn't limit my ability to send emails through RStudio using gmailr but does limit my ability in Cloud Run. I'm basing this guess? on the following:

• print(gm_scopes()) run remotely says I have full permission, I think, printing the below:

                                                  labels 
          "https://www.googleapis.com/auth/gmail.labels" 
                                                    send 
            "https://www.googleapis.com/auth/gmail.send" 
                                                readonly 
        "https://www.googleapis.com/auth/gmail.readonly" 
                                                 compose 
         "https://www.googleapis.com/auth/gmail.compose" 
                                                  insert 
          "https://www.googleapis.com/auth/gmail.insert" 
                                                  modify 
          "https://www.googleapis.com/auth/gmail.modify" 
                                                metadata 
        "https://www.googleapis.com/auth/gmail.metadata" 
                                          settings_basic 
  "https://www.googleapis.com/auth/gmail.settings.basic" 
                                        settings_sharing 
"https://www.googleapis.com/auth/gmail.settings.sharing" 
                                                    full 
                              "https://mail.google.com/" 

• when I run gm_oauth_app() both locally and remotely, it's returning the same thing, where below I'm replacing the actual key value with "my_hidden_key.apps.googleusercontent.com":

> gm_oauth_app()
<oauth_app> gmailr
  key:    my_hidden_key.apps.googleusercontent.com
  secret: <hidden>

• I also tried print(gm_profile(user_id = "daniel.sommerhauser@royals.com", verbose=TRUE)) both locally and remotely, and locally I get the below(*), but remotely I do get an error (**):

(*)

Logged in as:
  * email: daniel.sommerhauser@royals.com
  * num_messages: 21
  * num_threads: 21

(**)

Error in gmailr_GET("profile", user_id, class = "gmail_profile") : 
  Gmail API error: 403
  Request had insufficient authentication scopes.
Calls: print -> gm_profile -> gmailr_GET -> gmailr_query
Execution halted

I did a lot of reading around the last few days. For example, I went through the workflow described at #160 (https://developers.google.com/identity/protocols/oauth2/service-account#delegatingauthority) for the OAuth 2.0 Client IDs I'm using, but that didn't get me past the 403 error. I realize that's different from a Service Account but thought it didn't think it hurt to try. Candidly I think most R users out there are not authorization experts and thus don't have a great sense of the difference between API Keys, OAuth 2.0 Client IDs, & Service Accounts, and more explanation of the differences on the gmailr read-me would be super beneficial to many.

One thing I'll say is that the process of being forced to "create an App" by Google Cloud is a little perplexing for a user like myself who is simply trying to add an email to team members at the bottom of a daily analytics R script. Like Google is asking for a link to "the app" I'm creating, a link to the "privacy policy (of the app)" I'm creating, and in my case I don't have a special, task-specific privacy policy or website I can even link. So that's a little confusing to me, and frankly I'm not sure how I'm going to get around it to use gmailr on Cloud Scheduler, but it could be just me.

Again I greatly appreciate the reply and I'll post what I ultimately find out here so others can use if helpful.

[jennybc]

Can you please turn set the gargle_verbosity option to "debug"? And show me the output when you use gmailr in a fresh session? The key part is that I want to see the output while auth is initiating.

https://gargle.r-lib.org/articles/troubleshooting.html#gargle_verbosity-option

I still think you are getting auth'ed as the default service account in the Cloud Build setting, which explains what you are seeing.

[sommerhd-royals]

Hi @jennybc - thanks again for the reply. I've had the gargle_verbosity option set to debug:

> library(gargle)
> options(gargle_verbosity = "debug")
> print(gargle_verbosity())
[1] "debug"

I'm sorry if this is a dumb question but when you say "the output when auth is initiating", are you saying what gm_auth() is showing? It's showing the below both locally and remotely, where I'm replacing the text before the first period in the key with "my_hidden_key":

<oauth_app> gmailr
  key:    my_hidden_key.apps.googleusercontent.com
  secret: <hidden>

This is I think the relevant part of the code I'm using, where I am replacing some sensitive information with "my_hidden":

gm_auth_configure(key = "my_hidden_key.apps.googleusercontent.com",
                  secret = "my_hidden_secret")

gm_oauth_app()

print(gm_scopes())

library(gargle)
options(gargle_verbosity = "debug")
print(gargle_verbosity())

print("showing output from gm_auth(cache = '.secret') below here")
gm_auth(cache = ".secret")

print("showing output from gm_oauth_app() below here")
gm_oauth_app()

print("now running gm_auth() but with email, scopes, and cache specified")
gm_auth(
  email = gargle::gargle_oauth_email(),
  scopes = "full",
  cache = gargle::gargle_oauth_cache()
)


print("now running gm_profile() with user_id set to my gmail email and verbose = TRUE")
print(gm_profile(user_id = "daniel.sommerhauser@royals.com", verbose=TRUE))

And it's the last line that generates the 403 error, as does an attempt to run gm_send_message():

<oauth_app> gmailr
  key:    my_hidden_key.apps.googleusercontent.com
  secret: <hidden>
                                                  labels 
          "https://www.googleapis.com/auth/gmail.labels" 
                                                    send 
            "https://www.googleapis.com/auth/gmail.send" 
                                                readonly 
        "https://www.googleapis.com/auth/gmail.readonly" 
                                                 compose 
         "https://www.googleapis.com/auth/gmail.compose" 
                                                  insert 
          "https://www.googleapis.com/auth/gmail.insert" 
                                                  modify 
          "https://www.googleapis.com/auth/gmail.modify" 
                                                metadata 
        "https://www.googleapis.com/auth/gmail.metadata" 
                                          settings_basic 
  "https://www.googleapis.com/auth/gmail.settings.basic" 
                                        settings_sharing 
"https://www.googleapis.com/auth/gmail.settings.sharing" 
                                                    full 
                              "https://mail.google.com/" 
[1] "debug"
[1] "showing output from gm_auth(cache = '.secret') below here"
trying `token_fetch()`
trying `credentials_service_account()`
Error caught by `token_fetch()`:
Argument 'txt' must be a JSON string, URL or file.
trying `credentials_external_account()`
aws.ec2metadata not installed; can't detect whether running on EC2 instance
trying `credentials_app_default()`
trying `credentials_gce()`
[1] "showing output from gm_oauth_app() below here"
<oauth_app> gmailr
  key:    my_hidden_key.apps.googleusercontent.com
  secret: <hidden>
[1] "now running gm_auth() but with email, scopes, and cache specified"
trying `token_fetch()`
trying `credentials_service_account()`
Error caught by `token_fetch()`:
Argument 'txt' must be a JSON string, URL or file.
trying `credentials_external_account()`
aws.ec2metadata not installed; can't detect whether running on EC2 instance
trying `credentials_app_default()`
trying `credentials_gce()`
[1] "now running gm_profile() with user_id set to my gmail email and verbose = TRUE"
Error in gmailr_GET("profile", user_id, class = "gmail_profile") : 
  Gmail API error: 403
  Request had insufficient authentication scopes.
Calls: print -> gm_profile -> gmailr_GET -> gmailr_query
Execution halted

I truly thought I had the issue figured out when navigating to Security -> API Controls -> App Access Control and changing access of the "my_hidden_key" noted above Client ID to 'Trusted", but that didn't fix my issue either.

[jennybc]

The above output shows me that GCE auth is what's happening, which explains the problem.

[1] "showing output from gm_auth(cache = '.secret') below here"
trying `token_fetch()`
trying `credentials_service_account()`
Error caught by `token_fetch()`:
Argument 'txt' must be a JSON string, URL or file.
trying `credentials_external_account()`
aws.ec2metadata not installed; can't detect whether running on EC2 instance
trying `credentials_app_default()`
trying `credentials_gce()`
!!!!! RIGHT HERE GCE AUTH HAS SUCCEEDED !!!!!
!!!!! no call to credentials_user_oauth2() is ever made !!!!!

[1] "showing output from gm_oauth_app() below here"
...

so this seems to be an example of the problem reported in r-lib/gargle#187.

We need a way for you to indicate that you don't want to use GCE auth with gmailr.

This is technically possible today, but I've never made it particularly easy. But let's try it. Here's a clunky way to do it now:

library(gargle)

cred_funs_clear()

cred_funs_add(credentials_user_oauth2 = credentials_user_oauth2)

# do gm_auth() here

cred_funs_set_default()

The "credential function registry" is documented here:
https://gargle.r-lib.org/reference/cred_funs.html

The code above temporary makes only the "normal user OAuth flow" available, so that gmailr auths as the desired normal user. (Then sets things back to their default.)

Between this and r-lib/gargle#187, it's clear gargle needs to provide some nicer way to do this, but let's prove the point first.

[jennybc]

Confirming that the current best way to explicitly ensure that you don't get automatic auth with GCE is this:

gargle::cred_funs_add(credentials_gce = NULL)

This line would need to appear prior to anything that would explicitly or implicitly initiate gmailr auth.