14.5.12. crate_anon.nlp_manager.cloud_request_sender

crate_anon/nlp_manager/cloud_request_sender.py

Copyright (C) 2015, University of Cambridge, Department of Psychiatry. Created by Rudolf Cardinal (rnc1001@cam.ac.uk).

This file is part of CRATE.

CRATE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

CRATE is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with CRATE. If not, see <https://www.gnu.org/licenses/>.

CloudRequestSender class.

class crate_anon.nlp_manager.cloud_request_sender.CloudRequestSender(text_generator: Generator[Tuple[str, Dict[str, Any]], None, None], crinfo: CloudRunInfo, ifconfig: InputFieldConfig, report_every: int = 500, incremental: bool = False, queue: bool = True)[source]

Class to encapsulate a NLP request outbound to a cloud NLP server.

class State(value)[source]: Request state.

__init__(text_generator: Generator[Tuple[str, Dict[str, Any]], None, None], crinfo: CloudRunInfo, ifconfig: InputFieldConfig, report_every: int = 500, incremental: bool = False, queue: bool = True) → None[source]

Initialise class

Parameters:

text_generator – Generator that generates text strings from the source database. See crate_anon.nlp_manager.input_field_config.InputFieldConfig.gen_text().
crinfo – A crate_anon.nlp_manager.cloud_run_info.CloudRunInfo object.
ifconfig – An crate_anon.nlp_manager.input_field_config.InputFieldConfig object.
report_every – Report to the log every n requests.
incremental – Process in incremental mode (ignoring source records that have not changed since last time)?
queue – Queue the requests for back-end processing (rather than waiting for an immediate reply)?

send_requests(global_recnum: int) → Tuple[List[CloudRequestProcess], bool, int][source]

Sends off a series of cloud requests and returns them as a list. self._queue determines whether these are queued requests or not. Also returns whether the generator for the text is empty.

Return tuple is: requests, some_records_processed, global_recnum.