Posts

Searching for Every Email in Users Gmail Account

Gmail Search is a live search for your entire domain.
This allows G Suite Super admins to search for any piece of text in any email in any mailbox across the domain. As long as the email is not permanently removed from the bin folder, Gmail Search can find it.

Note: this process may take a long time if you are searching through many accounts at the same time.

To start a live search to find emails from one user’s account, click on Search through messages.

"gmail search" tab in GAT.

Now in the Query box enter some limiting search operators to narrow down the possible results which are returned. In the below example I made sure to search through all folder structures across Gmail by using in:anywhere but I made sure to ignore Google Meet/Hangouts chat records by using -is:chat. The (-) symbol excludes chat messages.

"gmail searches" tab.


Enter which user you want to search through in the fields provided below the Query box. I searched through one users account. You can search through a whole group of users by entering a Google Group. Or you can search through an Org Unit and its Sub-orgs.

Now navigate the Recent tab, when the search has finished you can click on the green button, to show the result.

"gmail searches" tab in gat+

Once the results are displayed you can explore all the results returned.

GAT's "gmail search" tab.

Once the results are displayed you can explore all the results returned.

There are many things you can do from this point forward.

  • Send a request to your Security Officer to have permission to view/download and delete these emails
  • You can export the metadata to a Google Spreadsheet or CSV

How to Find All Emails for a Gmail User

When GAT+ is installed, our system begins to index all of the emails in every account covering a period of 28 days (4 weeks) prior to the install date. This helps us build up some statistics so you can view recent trends. We then index every email going forward indefinitely.

In cases where you need to search for emails older than 28 days from the date of GAT+ install you can use the real-time search called Gmail Search in GAT+.

Search the entirety of any users mailbox for any set of emails, from any time period, as long as the email is still there (not permanently deleted by the user).

Search the entirety of any users mailbox for any set of emails, from any time period, as long as the email is still there (not permanently deleted by the user).
slect "new" in the gmail searches tab

You can exclude “chats” if you use “in:anywhere -in:chats” if you wish to narrow down the search to a specific period use the following search operators after:YYYY/MM/DD and/or before:YYYY/MM/DD. Alternatively, you can use older_than:5d or newer_than:30d.

So the full search term might look like this “in:anywhere -in:chats after:2019/03/01 before:2019/03/31 is:read”. View the full list of search operators available.


The search may take quite some time especially if you’re dealing with thousands of emails.

When the Start Search button is pressed, this will redirect you to the Recent tab. In this tab, you will see the status of all your email searches.

When the Start Search button is pressed, this will redirect you to the Recent tab. In this tab, you will see the status of all your email searches.

After the search completes, you can select the green check mark and all emails for this user will be displayed.

After the search completes, you can select the green check mark and all emails for this user will be displayed.

Once the results are shown, you can add and build new filters on the top of the current search. To find specific emails or examine the totality of that user’s activity.

Once the results are shown, you can add and build new filters on the top of the current search. To find specific emails or examine the totality of that user’s activity.

To add additional filters on top of this real-time search, click on the Apply custom filter button.

To add additional filters on top of this real-time search, click on the Apply custom filter button.

One example of using the Apply custom filters in Gmail Search is to narrow down the above search to find only emails with more than 2 email attachments.

One example of using the Apply custom filters in Gmail Search is to narrow down the above search to find only emails with more than 2 email attachments.

You can always return to Gmail Searches you had previously done and remove them from the listing.

You can always return to Gmail Searches you had previously done and remove them from the listing.

In conclusion, Gmail Search provides a powerful alternative to scan based searches but may be slower as the email metadata is not already indexed. If your email audit does not require up to the minute information I would recommend sticking with scan based searches within the Emails tab.

In conclusion, Gmail Search provides a powerful alternative to scan based searches but may be slower as the email metadata is not already indexed. If your email audit does not require up to the minute information I would recommend sticking with scan based searches within the Emails tab.

How to Debug Email Routing Problems

GAT+ Lets Admins see the filters Users have in place. Useful for debugging email routing problems. Users audit, Emails, click on the filters count for the User of interest to see what the underlying filters are.

GAT Gmail filter

 

Understanding Group Activity email and file sharing

Some recent questions raise the interesting problem of how do we measure email activity at group level (and group activity in general).

Groups inside Google are phenomenally interesting and at GAT we spend quite a bit of time thinking about them. At a simple level, a group can act as an email alias for a function, ‘sales@generalaudittool.com’.  A group can also form a platform for email collaboration, ‘new-ideas@generalaudittool.com’. It can also form the basis of a mini-community, ‘development@generalaudittool.com’, where the group name can be used as the basis for sharing documents, emails, access rights, etc.

In reality, any group once established, tends to function to varying degrees in all three roles inside the organization. In addition, you have the situation that members of one group are often members of other groups. Looking at the output of the members, you can never tell if the mail is group related.

However, with GAT+Email you can see some important details

1) How busy, in general, members of that group have been.

Under the ‘Users’ audit, select the email tab, the search for the group you are interested in, then, clicking on ‘show graph’ you can see the detailed internal and external email loads for members of that group.

Understanding Group Activity email and file sharing

 

2) The amount of email sent specifically to any Group (where the group is viewed as a collection of members), which domain and which user it came from (internal or external).

Here we use GAT+Email and select the email audit. Once in the email audit we search for the group we are interested in. This can be done two ways, firstly we can just reference the Group and this will return all emails for which members of the group were involved

Or we can make the group address the subject of a specific search, (the results show only the email sent specifically to the group address).

Here we can see in the table everyone inside and outside the organization involved in sending email to that group address and the volume of email involved. Domain or individual can be turned on or off to see the values for each category more clearly.

In addition, you can search for all the emails sent ‘from’ a particular group address and then using the ‘Recent Filters’ tab combine the two searches to produce a combined set of results. Again these can be examined in the Sender/Receiver table.

A really powerful exercise is to limit the searches to a date range, then look at the ‘Domain Connections’ table for the results. At the top of the table select ‘Show communication graph’.

See this post. The graph shows newest relationships on the right, oldest on the left. Domains sending to us on the top, domains we send to on the bottom. In between have emails in both directions, depending on the balance.

3) To what degree that group is directly involved in document sharing and collaboration

In this case, we go to the ‘Group’ audit on the homepage.

In this audit we can search for a group of interest and from there we can see details like the total of the overall percentage of docs created by members of this group, the percentage of shared in documents shared to its members and the percentage of shared out docs, shared out by its members.

In addition, you can see details like the number of group documents (those shared), the number shared explicitly to the group address and the numbers shared by members of the group to one another. This helps you understand if the Group is being used for Collaboration.

 

Understanding and Measuring Internal Email Communications

GAT supports two ways to measure and understand internal email communications. The first method is through the ‘Users Audit’.

Select the ‘Users Audit’ from the GAT homepage, and from the range of audit areas, select the ‘Emails’ tab (1).  This shows the last 30 days of email activity, both internal and external.

By default, this will show all the users on the domain. As this might be a large number of users you might like to focus on an individual or a group. You can enter an individual or a group name at (2). You can also select an Organization Unit (3).

The email volumes for each user is laid out in 8 columns. The first 4 are for external volumes and the last 4 are for internal volumes. The columns show the following

No. of email sent

No. of emails sent with files attached

No. of emails received

No. of emails received with files attached

For internal emails look to the second set of 4 columns (5).

Clicking on any number in blue shows you the email metadata that is behind the figure. You can see the ‘from’, ‘to’, ‘cc’ etc.

If you wish to see the volumes for each day of the last 30 days, select ‘Show daily stats’ (6).

All columns are sortable.

They say a picture is worth a thousand words, so for the big picture select the ‘Business Intelligence’ section from the GAT homepage.

From there select ‘Internal/External Collaboration’.

Then select a look back period of the last 4 weeks.

Select ‘Show Graph’.

Wait for the graph to draw. This will take about 30 seconds.

When the graph has drawn select the following choices. Under ‘View’ select ‘Emails’ and in the Filters section, select ‘Dampen external collaboration’

What you are left with on the graph is a map of the actual internal email activity for ‘replied to’ email over the last four weeks.

If you want to see the numerical values behind the pairs of email relationships, select ‘Pairs’. You can sort to the top pairs to the top.

You can even explore the activity between any two individuals, any two groups, any two OU’s or any combination of the above.

Finally, if you want to understand more about this suite of tools and how you can make them work for you, you can read here.

Notes

The two methods are actually slightly different in terms of what they measure. Under the ‘Users Audit’ absolute volumes of emails are being measured. Every email sent and every email received is counted and tabulated. Under the ‘Business Intelligence’ graph, the measurement only reflects ‘replied to’ emails. Emails not replied to or ignored are not counted for use in this graph as they are not considered to form part of a collaborative effort. The purpose of the Collaboration graph is to show collaboration activity.

Visualizing specific G Suite relationships or workloads

Specific Use Case

Specific use cases for these features include decommissioning users, analysing relationships, understanding the state of a business relationship with a specific company, identifying contacts with a specific company, assessing workload and many other important Business Intelligence tasks.

Read more

How to Schedule Reports for Top 10 or More Email Senders and Receivers

This post will show you how GAT+ can generate a report of all email activity for each user. The report can be scheduled daily, weekly or monthly and will be generated in the form of a spreadsheet that can be sorted by the desired column. The report will cover both internal and external email.

To start building this scheduled report the Admin should go to the Users audit first select the group or OU to report on, then select the Emails tab, followed by clicking the Show daily stats link

From there, select the ‘Schedule’ link

At this point, the Admin will be able to choose the exact type of report s/he wants.

For the daily report, select as a schedule ‘Every day – after midnight’ or choose a custom Cron period. You can also choose the report type such as ‘New spreadsheet will be created for each day’.

You may also add additional email recipients for the report.

You can now also select both the number of results you want, the order in which you want them sorted and select from a wide range of sort fields. 

Once you have made the selection, press ‘Update’ and the job will become a scheduled job for your domain. A new spreadsheet will be created every day, cover the previous day. The Admin will be emailed a link of the spreadsheet.

 

An example of the spreadsheet output looks like this.

Freeze row 1 and then select any column of interest,  finally sort the sheet based on that column in the Z to A order to bring the largest numbers to the top.

Here is an example of a calendar monthly aggregate report

You might also find these posts relevant:  

Demonstrating GAT+’s Powerful Reporting Range

4 Great Scheduled Audits You Should Set in GAT

Scheduled Report for Top 10 (and more) Email Senders and Receivers (by emails, by attachments, by bytes)

Report Files Created in the Last 24 Hours

 

Content Compliance Testing in Gmail

Google now provide in-line genuine Data Loss Prevention (DLP) for Gmail and Drive. In this post, we show you how to configure DLP for G Suite eMail. We now have a DLP reporting section in GAT+.

From the G Suite for Work Admin console it is now possible to test email contents for compliance with company rules and regulations or to perform actions like alerting a Security officer if a credit card number is detected passing through. This is potentially a very important link in your cloud security fence and complementary to tests and checks performed by the General Audit Tool.

From the Admin console, select Apps, Google Apps, Settings for Gmail and then scroll down to ‘Advanced settings’.

Apps > Google Apps > Settings for Gmail > Advanced settings

Under ‘General Settings’ you will find three configuration areas, each linked to compliance. In this example, we are going to look at ‘Content compliance’, which is where we can execute a regular expression test on email contents. (Here are some regular expressions you can use)

The configure button for each section appears on the right of the screen, click on this to configure. 

You can ‘Add Setting’ to Inbound or Outbound email for your domain or to internal email.

We selected both Inbound and Outbound to test for Credit Card movement inside email.

Under ‘Expressions we select ‘Advanced Content match’

Gmail settings

For Location, we select the Body of the message and for Match type, we select Matches regex. A nice feature is you can test the expression before saving it.

Gmail settings

\b(1800|2131|30[0-5]\d|3[4-7]\d{2}|4\d{3}|5[0-5]\d{2}|6011|6[2357]\d{2})[- ]?(\d{4}[- ]?\d{4}[- ]?\d{4}|\d{6}[- ]?\d{5})\b

The regular expression used to test for credit cards is given above – cut and paste to your own configuration.

Be sure to save the ‘Expression’ test you have just built (this is the first of 3 save actions you will have to take).

Step 3 will allow you to configure what actions you wish to take if a match is found, these can include sending a copy of the message to a security officer, or quarantining the message until further release after inspection.

more Gmail settings

then Save the completed compliance rule

and finally, Save the Configuration!

Note also the rule may take up to 1 hour before coming into effect.

A Google Help Page on this feature is to be found here.

Measuring User email Response Times

GAT+ allows Admins to monitor turnaround time on all email conversations. This can be useful for monitoring the performance of a group like Sales or Support, or for resolving a question of how quickly an issue was dealt with and at what point the delays occurred.

For any given search criteria against email metadata GAT+ also produces 3 sets of ‘quick’ filters.

These cover emails sent out from your domain, emails sent into your email and internal emails. The three quick filters have now been extended to include a count of emails that have replies in the filter time range covered and also an average of the response time for each collection of replies. In ‘red’ the number of replies is counted along with the average time it took a member of your domain to make the reply. In ‘orange’ you can see the number of replies by external users to emails sent from your domain, it also includes the average time these external responders took to reply. Finally, in ‘yellow’ you have the number of internal emails, including emails that are replies and the average response time for those replies.

Admins can click the ‘replies’ count in any one of these three quick filters to explore each conversation in more detail.

At this level, Admins can see the top level conversations for each email send/receive. From here you can identify all the participants, the number of threads in any given conversation and the average response time across all the threads (Reply Delay column).

Clicking on the thread count lets Admins explore each conversation in more detail

Now the Admin can see the full set of conversation threads and the time each party took to respond to a previous part of the thread. Moving the mouse over each reply delay will show the Admin which thread this delay refers to. In most cases, the delay refers to the time to respond to the previous email. However, in some cases, the participants might decide to go back to an earlier part of the conversation and reply to that part of the thread. GAT correctly identifies which of the earlier thread parts this delay in response refers to.

As with all GAT reports the Admin can export the results from any stage of the filtering process to a spreadsheet.

Where emails are part of ‘reply-to’ conversations the reply delay for each part is reported in seconds. This data may be used for subsequent post-processing by the Domain Admin.

 

Logging All Email to or from a Specific Address

A common task for Admins and Auditors is to keep a daily log for emails to specific addresses.

The log can be used to determine workload or communication monitoring.

GAT makes this task simple.

Let us look at an example of a daily log of email sent directly to ‘support’

This example will find only email with ‘support’ in the ‘Sent To:’ field, sometimes ‘support’ can be cc’ed or Bcc’ed, if you want to record these too, select ‘Any Recipient’

You will see you have a selection of search filters in the ‘General Search’, these are

Subject Select emails where the subject matches the string
Any Email Select where the search address is in any position
From Select where the search address is in the ‘From:’ position
Any Recipient Select where the search address is in any recipient position
Sent To Select where the search address is in the ‘Sent To:’ position
CC Select where the search address is in the ‘cc:’ position
BCC Select where the search address is in the ‘bcc:’ position
Filename Select where the search line matches any attached file name

Having set the selection criteria, we next select the date, as this is a daily log we start with today’s date. Clicking on ‘Search’ will return all the emails that match the criteria already sent today. If no matches are returned it does not matter, we have now built our audit policy which we are going to schedule.

Next click on ‘Schedule/Save’ – this will take us through to the scheduler.

You can see the search policy is automatically carried through. You can select the run period, in this case, we selected ‘every day after midnight’ (this will be midnight your time if you set your timezone in GAT).

GAT is clever enough to automatically move the start date for each new report by the amount of time in the period you select. So if you select a day, the next start date will be today’s date+1. If it was a weekly schedule you picked it will be today’s date+7.

You can also select a user(s), or a group(s) – comma separated – to get copies of the report.