Edit on GitHub

datasources.tumblr

Initialize Tumblr data source

 1"""
 2Initialize Tumblr data source
 3"""
 4
 5# An init_datasource function is expected to be available to initialize this
 6# data source. A default function that does this is available from the
 7# backend helpers library.
 8from common.lib.helpers import init_datasource
 9__all__ = ("init_datasource",)
10
11# Internal identifier for this data source
12DATASOURCE = "tumblr"
13NAME = "Tumblr"
def init_datasource(database, logger, queue, name, config): 
40def init_datasource(database, logger, queue, name, config):
41    """
42    Initialize data source
43
44    Queues jobs to scrape the boards that were configured to be scraped in the
45    4CAT configuration file. If none were configured, nothing happens.
46
47    :param Database database:  Database connection instance
48    :param Logger logger:  Log handler
49    :param JobQueue queue:  Job Queue instance
50    :param string name:  ID of datasource that is being initialised
51    :param config:  Configuration reader
52    """
53    pass

Initialize data source

Queues jobs to scrape the boards that were configured to be scraped in the 4CAT configuration file. If none were configured, nothing happens.

Parameters
  • Database database: Database connection instance
  • Logger logger: Log handler
  • JobQueue queue: Job Queue instance
  • string name: ID of datasource that is being initialised
  • config: Configuration reader