Schedule Job history for emails - where is that stored?

A very common administrative request is to to view historical audits of scheduled jobs, such as emails that are sent on a daily or weekly basis.

The arcadia Jobs page provide a central management console into creating new schedules, viewing existing jobs and deleting / pausing.

However, the only history that’s provided is only for the most recent run of the job. In this case we can see the JobID=27 was last run on Aug 18.

To get longer history of jobs, you can look in the arcadia visualization server metastore - specifically the jobs_joblog table.

  • image

You can query this table by searching for the specific job id (which in our previous example was 27)

    select * from jobs_joblog where schedule_id=27 order by start_time desc limit 5;

This will provide full history of that job.

| 2228 | 2018-08-17 19:39:51 | 2018-08-17 19:40:31 | {"run_parameters": {"skip_next_targets": false, "force_run": true, "creator_id": 1, "creator_username": "admin", "editor_id": 1, "editor_username": "admin", "image": "/tmp/tmpk_saj61v.png", "csv_names_to_filepath": {"516": "/tmp/tmpgwtfd3rv.csv", "<<Order Date String.start>>515": "/tmp/tmprrkkbrbq.csv"}, "error": null, "image_type": "E,A,D,C", "url_params": "embed=true&sheet=1&param.Order%20Date%20String.start=2011-09-01&param.Order%20Date%20String.end=2018-02-28&param.Order%20Date%20String.config=%7B%22.rel%22%3A%7B%22startDate%22%3A%222011-09-01%22%2C%22endDate%22%3A%222018-02-28%22%2C%22relativeType%22%3A%22previous%22%2C%22lastCount%22%3A%2278%22%2C%22datetimeFormat%22%3A%22YYYY-MM-DD%22%2C%22anchorType%22%3A%22current%22%2C%22unit%22%3A%22Month%22%7D%7D&scope=%5B%7B%22filterId%22%3A514%2C%22datasetId%22%3A21%2C%22filterName%22%3A%22Order%20Date%20String%22%7D%5D", "to_addresses": ["shaun@xxxx.com"], "visual_url": "http://10.0.0.221:38888/arc/apps/app/517", "location_pathname": "/arc/apps/app/517", "email_template": "", "email_subject": "[ArcadiaData] Test parameters", "message": "", "cc_addresses": [], "app_id": 517, "from_address": "admin <emailscheduler2018@gmail.com>", "name": "Test_parameters.pdf.png", "start_time": "08/17/18 19:39:51 UTC", "run_time": "0:00:37.655980", "run_interval": "Hourly", "run_interval_cron": "15 0 * * *", "next_run_time": "08/18/18 00:15:00 UTC"}, "1": {"image_type": "E,A,D,C", "url_params": "embed=true&sheet=1&param.Order%20Date%20String.start=2011-09-01&param.Order%20Date%20String.end=2018-02-28&param.Order%20Date%20String.config=%7B%22.rel%22%3A%7B%22startDate%22%3A%222011-09-01%22%2C%22endDate%22%3A%222018-02-28%22%2C%22relativeType%22%3A%22previous%22%2C%22lastCount%22%3A%2278%22%2C%22datetimeFormat%22%3A%22YYYY-MM-DD%22%2C%22anchorType%22%3A%22current%22%2C%22unit%22%3A%22Month%22%7D%7D&scope=%5B%7B%22filterId%22%3A514%2C%22datasetId%22%3A21%2C%22filterName%22%3A%22Order%20Date%20String%22%7D%5D", "to_addresses": ["shaun@xxx.com"], "visual_url": "http://10.0.0.221:38888/arc/apps/app/517", "location_pathname": "/arc/apps/app/517", "email_template": "", "email_subject": "[ArcadiaData] Test parameters", "message": "", "cc_addresses": [], "app_id": 517}} | C | job/email/Test_parameters_uwtWLoG | | Finished Task #2: email job target (id #29) | 27 | 1 |

| 2194 | 2018-08-17 00:15:59 | 2018-08-17 00:16:12 | {"run_parameters": {"skip_next_targets": false, "force_run": null, "creator_id": 1, "creator_username": "admin", "image": "/tmp/tmpk21923fx.png", "csv_names_to_filepath": {"516": "/tmp/tmp1tf5_n7p.csv", "<<Order Date String.start>>515": "/tmp/tmp9hloxqll.csv"}, "error": null, "image_type": "E,A,D,C", "url_params": "embed=true&sheet=1&param.Order%20Date%20String.start=2011-09-01&param.Order%20Date%20String.end=2018-02-28&param.Order%20Date%20String.config=%7B%22.rel%22%3A%7B%22startDate%22%3A%222011-09-01%22%2C%22endDate%22%3A%222018-02-28%22%2C%22relativeType%22%3A%22previous%22%2C%22lastCount%22%3A%2278%22%2C%22datetimeFormat%22%3A%22YYYY-MM-DD%22%2C%22anchorType%22%3A%22current%22%2C%22unit%22%3A%22Month%22%7D%7D&scope=%5B%7B%22filterId%22%3A514%2C%22datasetId%22%3A21%2C%22filterName%22%3A%22Order%20Date%20String%22%7D%5D", "to_addresses": ["shaun@xxxx.com"], "visual_url": "http://10.0.0.221:38888/arc/apps/app/517", "location_pathname": "/arc/apps/app/517", "email_template": "", "email_subject": "[ArcadiaData] Test parameters", "message": "", "cc_addresses": [], "app_id": 517, "from_address": "admin <emailscheduler2018@gmail.com>", "name": "Test_parameters.pdf.png", "start_time": "08/17/18 00:15:59 UTC", "run_time": "0:00:10.421234", "run_interval": "Hourly", "run_interval_cron": "15 0 * * *", "next_run_time": "08/18/18 00:15:00 UTC"}, "1": {"image_type": "E,A,D,C", "url_params": "embed=true&sheet=1&param.Order%20Date%20String.start=2011-09-01&param.Order%20Date%20String.end=2018-02-28&param.Order%20Date%20String.config=%7B%22.rel%22%3A%7B%22startDate%22%3A%222011-09-01%22%2C%22endDate%22%3A%222018-02-28%22%2C%22relativeType%22%3A%22previous%22%2C%22lastCount%22%3A%2278%22%2C%22datetimeFormat%22%3A%22YYYY-MM-DD%22%2C%22anchorType%22%3A%22current%22%2C%22unit%22%3A%22Month%22%7D%7D&scope=%5B%7B%22filterId%22%3A514%2C%22datasetId%22%3A21%2C%22filterName%22%3A%22Order%20Date%20String%22%7D%5D", "to_addresses": ["shaun@xxxx.com"], "visual_url": "http://10.0.0.221:38888/arc/apps/app/517", "location_pathname": "/arc/apps/app/517", "email_template": "", "email_subject": "[ArcadiaData] Test parameters", "message": "", "cc_addresses": [], "app_id": 517}} | C | job/email/Test_parameters_SuY8PmJ | | Finished Task #2: email job target (id #29) | 27 | 1 |

The “contents_archive” field will have everything pertaining to that email, recipients , actual message, what dashboard, etc. It’s string field that’s formatted at JSON, here’s an example:

{
    "1": {
        "url_params": "embed=true&sheet=1&param.Order%20Date%20String.start=2011-09-01&param.Order%20Date%20String.end=2018-02-28&param.Order%20Date%20String.config=%7B%22.rel%22%3A%7B%22startDate%22%3A%222011-09-01%22%2C%22endDate%22%3A%222018-02-28%22%2C%22relativeType%22%3A%22previous%22%2C%22lastCount%22%3A%2278%22%2C%22datetimeFormat%22%3A%22YYYY-MM-DD%22%2C%22anchorType%22%3A%22current%22%2C%22unit%22%3A%22Month%22%7D%7D&scope=%5B%7B%22filterId%22%3A514%2C%22datasetId%22%3A21%2C%22filterName%22%3A%22Order%20Date%20String%22%7D%5D",
        "cc_addresses": [],
        "location_pathname": "/arc/apps/app/517",
        "email_template": "",
        "image_type": "E,A,D,C",
        "app_id": 517,
        "to_addresses": [
            "shaun@xxxx.com"
        ],
        "visual_url": "http://10.0.0.221:38888/arc/apps/app/517",
        "email_subject": "[ArcadiaData] Test parameters",
        "message": ""
    },
    "run_parameters": {
        "url_params": "embed=true&sheet=1&param.Order%20Date%20String.start=2011-09-01&param.Order%20Date%20String.end=2018-02-28&param.Order%20Date%20String.config=%7B%22.rel%22%3A%7B%22startDate%22%3A%222011-09-01%22%2C%22endDate%22%3A%222018-02-28%22%2C%22relativeType%22%3A%22previous%22%2C%22lastCount%22%3A%2278%22%2C%22datetimeFormat%22%3A%22YYYY-MM-DD%22%2C%22anchorType%22%3A%22current%22%2C%22unit%22%3A%22Month%22%7D%7D&scope=%5B%7B%22filterId%22%3A514%2C%22datasetId%22%3A21%2C%22filterName%22%3A%22Order%20Date%20String%22%7D%5D",
        "csv_names_to_filepath": {
            "516": "/tmp/tmpw1t6yh.csv",
            "<<Order Date String.start>>515": "/tmp/tmpsw_PW_.csv"
        },
        "from_address": "admin <emailschedule@xxxx.com>",
        "image": "/tmp/tmpWiZ6dk.png",
        "app_id": 517,
        "run_interval_cron": "15 * * * *",
        "message": "",
        "image_type": "E,A,D,C",
        "force_run": true,
        "skip_next_targets": false,
        "location_pathname": "/arc/apps/app/517",
        "email_template": "",
        "start_time": "03/09/18 15:49:13 UTC",
        "editor_username": "admin",
        "to_addresses": [
            "shaun@xxx.com"
        ],
        "visual_url": "http://10.0.0.221:38888/arc/apps/app/517",
        "run_time": "0:00:36.327202",
        "creator_username": "admin",
        "editor_id": 1,
        "next_run_time": "03/09/18 16:15:00 UTC",
        "cc_addresses": [],
        "name": "Test_parameters.pdf.png",
        "creator_id": 1,
        "error": null,
        "email_subject": "[ArcadiaData] Test parameters",
        "run_interval": "Hourly"
    }
}

HI Shaun,

Suppose if my scheduled job fails because of some error, can i get an email alert if my job fails in Arcadia, if yes then kindly let me know.

@Purnesh yes that feature is coming in a future release.

Thank you very much Tadd

HI Tadd,

This even applicable to the threshold based email also right?

Like i have created a threshold based email and the job has failed because of some error, so in future version of Arcadia i’ll be getting an email regarding the failure status right.

Am i correct? please let me know

@Purnesh yes, if there is an internal error during the process of the scheduled email job.