14.5.12. crate_anon.nlp_manager.cloud_request_sender
crate_anon/nlp_manager/cloud_request_sender.py
Copyright (C) 2015, University of Cambridge, Department of Psychiatry. Created by Rudolf Cardinal (rnc1001@cam.ac.uk).
This file is part of CRATE.
CRATE is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
CRATE is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with CRATE. If not, see <https://www.gnu.org/licenses/>.
CloudRequestSender class.
- class crate_anon.nlp_manager.cloud_request_sender.CloudRequestSender(text_generator: Generator[Tuple[str, Dict[str, Any]], None, None], crinfo: CloudRunInfo, ifconfig: InputFieldConfig, report_every: int = 500, incremental: bool = False, queue: bool = True)[source]
Class to encapsulate a NLP request outbound to a cloud NLP server.
- __init__(text_generator: Generator[Tuple[str, Dict[str, Any]], None, None], crinfo: CloudRunInfo, ifconfig: InputFieldConfig, report_every: int = 500, incremental: bool = False, queue: bool = True) None [source]
Initialise class
- Parameters:
text_generator – Generator that generates text strings from the source database. See
crate_anon.nlp_manager.input_field_config.InputFieldConfig.gen_text()
.crinfo – A
crate_anon.nlp_manager.cloud_run_info.CloudRunInfo
object.ifconfig – An
crate_anon.nlp_manager.input_field_config.InputFieldConfig
object.report_every – Report to the log every n requests.
incremental – Process in incremental mode (ignoring source records that have not changed since last time)?
queue – Queue the requests for back-end processing (rather than waiting for an immediate reply)?
- send_requests(global_recnum: int) Tuple[List[CloudRequestProcess], bool, int] [source]
Sends off a series of cloud requests and returns them as a list.
self._queue
determines whether these are queued requests or not. Also returns whether the generator for the text is empty.Return tuple is:
requests, some_records_processed, global_recnum
.