Edit on GitHub

datasources.tumblr

Initialize Tumblr data source

 1"""
 2Initialize Tumblr data source
 3"""
 4
 5# An init_datasource function is expected to be available to initialize this
 6# data source. A default function that does this is available from the
 7# backend helpers library.
 8from common.lib.helpers import init_datasource
 9__all__ = ("init_datasource",)
10
11# Internal identifier for this data source
12DATASOURCE = "tumblr"
13NAME = "Tumblr"
def init_datasource(database, logger, queue, name, config): 
39def init_datasource(database, logger, queue, name, config):
40    """
41    Initialize data source
42
43    Queues jobs to scrape the boards that were configured to be scraped in the
44    4CAT configuration file. If none were configured, nothing happens.
45
46    :param Database database:  Database connection instance
47    :param Logger logger:  Log handler
48    :param JobQueue queue:  Job Queue instance
49    :param string name:  ID of datasource that is being initialised
50    :param config:  Configuration reader
51    """
52    pass

Initialize data source

Queues jobs to scrape the boards that were configured to be scraped in the 4CAT configuration file. If none were configured, nothing happens.

Parameters
  • Database database: Database connection instance
  • Logger logger: Log handler
  • JobQueue queue: Job Queue instance
  • string name: ID of datasource that is being initialised
  • config: Configuration reader