Pharmacovigilance is vital for post-market drug safety monitoring. Traditional trials inadequately capture adverse reactions. Patient-generated opinions offer valuable insights but pose challenges due to limited availability, inefficient annotation scheme, and volume of drug reviews, indicating the need of automation. To address the challenge posed by the limited availability of annotated drug reviews, we introduce a novel mechanism of annotating drug reviews dataset through human intervention, specifically tailored for Aspect Term Extraction (ATE) and Polarity Detection (PD) in three medical conditions: Depression, Arthritis and Birth control, thereby making it publicly available to facilitate future research. We first designed, ATEdrug, an automated annotation scheme with minimal human intervention by employing an expert-driven rule-based approach. We further deploy ATEdrug for sequence labelling and classification tasks in drug reviews. The automatically labeled data is further used for training and testing of deep learning models: BERT, BioBERT, and ClinicalBERT, for clinical use. We manually evaluate the results through annotator agreement to validate the effectiveness of ATEdrug. The labeled dataset is further used to construct transformer-based models. Our proposed model is reliable, safe and trustworthy for healthcare domain, thereby eliminating the compromises with hallucination and data security through generative AI models. All experimental codes and data are available on GitHub (https://github.com/CMOONCS/ateDrug.git).