Publication date: April 07, 2025
LAWYERS OF KIELTYKA GLADKOWSKI KG LEGAL TAKE PART IN TRAINING ORGANISED BY LIFE SCIENCE CLUSTER, on the example of SeniorAngel.app project
Artificial intelligence has become an integral part of modern medicine. Its use in diagnostics, therapy and healthcare system management is revolutionizing the way patients are diagnosed and treated. Thanks to increasingly advanced algorithms and systems based on machine learning, AI contributes to improving the quality of medical care, increasing efficiency and reducing costs. Artificial intelligence (AI) plays a key role in medical diagnostics, supporting doctors in diagnosing diseases based on medical data, images (such as X-rays, magnetic resonance imaging) and laboratory test results. Thanks to algorithms for analyzing medical images, AI can detect diseases such as cancer, heart disease or neurological changes. AI also supports the personalization of treatment, analyzing patient data such as medical history, genetic data or lifestyle to adapt therapy to individual needs, which improves its effectiveness. AI algorithms can also predict the risk of diseases based on population and historical data, allowing the implementation of preventive measures. In addition, AI supports the optimization of hospital processes, such as managing schedules, drug stocks or forecasting the need for hospital beds. AI is also used to monitor patients’ health, analyzing data from wearable devices such as smartwatches or biometric sensors to monitor parameters such as blood pressure, heart rate, and glucose levels, allowing for early detection of health changes. AI also supports medical data management, facilitating their storage, analysis, and processing, which improves access to medical databases and facilitates collaboration between specialists. In the area of telemedicine, AI supports remote care services, helping doctors diagnose and monitor patients’ health remotely.
The Digitalization Committee approved a project aimed at developing artificial intelligence and e-services in health, financed from the National Reconstruction Plan. PLN 1.2 billion has been allocated for the implementation of this project. The aim of the investment is to improve the quality of diagnostics and facilitate access to specialists, as well as to improve the work of medical professionals.
The project assumes the development of telemedicine, which will enable remote care for patients, increasing the availability of medical services, especially in less urbanized areas. Another important element is the creation of a central repository of medical data, which will allow easier and faster access to patients’ health histories, and will also improve the exchange of information between medical facilities. The next step is the digitization of medical records, which will reduce the time spent on administration, giving doctors more precise diagnostic tools, and patients – faster access to consultations and specialists.
These investments are aimed at improving the efficiency of the healthcare system by introducing modern solutions that both support the work of medical professionals and enable patients to have easier access to the care they need.
The training that our lawyers participated concerned the presentation of modern support for senior care using an artificial intelligence system. This product is SeniorAngel.app. The project is implemented by KPCOM in consortium with the Krakow University of Technology.
This is an innovative application that revolutionizes the way of caring for the elderly, while enabling their loved ones, caregivers and social care units to organize activities more effectively and improve the quality of life of seniors using AI technology. A smartphone application that helps 9 million seniors in Poland (50 million in the EU) suffering from dementia and neurodegenerative diseases. By using smartphones, we eliminate the need to purchase expensive equipment, enabling detailed analysis of movements, detection of potential illnesses, falls and irregularities in the rhythm of the day. Problems related to caring for seniors are becoming more and more severe, both for families and the social care system. The cost of employing a senior carer, amounting to PLN 3-8 thousand per month, is a financial barrier for many families, which means that the availability of appropriate care is becoming limited. Meanwhile, seniors requiring constant care constitute about 30% of the population, and their number increases with age, which deepens the problem, while the cost of the application is EUR 10-20 / month
The application monitors the health of seniors, supports their safety and ensures regular medication and hydration. Functions such as remote monitoring of activity, analysis of daily rhythm or fall detection help to quickly respond to potential threats. The project brings two main benefits, which are improving the quality of life of caregivers of the elderly, thanks to remote monitoring and automation – it replaces the caregiver. Additionally, it brings financial savings for care units, thanks to automation, task optimization and personalization, in the order of 37% of personnel costs. The product is to be available on the market from January 2026.
Private nursing homes face serious difficulties, such as a lack of staff, high costs and low efficiency, which often make them inaccessible to people with lower incomes. Local governments, which are responsible for organizing senior care, are faced with an overloaded system – the number of seniors is too large and the available resources are insufficient. In addition, there is a lack of effective tools for deinstitutionalization, which makes it difficult to transform traditional forms of care into more flexible and effective methods.
The lack of automation in senior care is a common problem that affects all institutions involved in senior care. Traditional forms of care are unable to meet the growing demand, making the situation increasingly difficult, especially in the context of an aging society.
The process of ageing of society, which also concerns Poland, is becoming a serious demographic challenge. Demographic forecasts of the Central Statistical Office indicate that in 2050 the share of people over 65 in Poland will be as much as 32.7% of the population. This creates the need to implement new solutions in senior care, such as automation, innovative technologies or the development of deinstitutionalization systems, to ensure an appropriate quality of life for older people in the face of these challenges.
The mass automation in senior care so far has mainly been SOS wristbands, but many seniors who require care do not use these devices. The reason is that they do not understand their operation, forget to put them on or charge them. The conclusion is that senior care must be ambient – that is, gentle, not requiring any activity on the part of the senior, but at the same time effective.
The proposed product is an innovative solution that enables discreet, 24/7 monitoring of seniors. It uses a system that displays the senior’s skeleton and collects important health parameters. Importantly, this system does not violate the privacy of seniors because it does not record video. Images are transformed into spatial points, and the data sent is anonymous, compliant with GDPR and other legal regulations, which ensures full privacy protection and no need for additional consents.
System operation:
The savings generated by the system result from the automation of many processes related to senior care, which allows for a significant reduction in costs and increased efficiency.
Dash1 Group – Senior
Home care devices:
Dash2 Group – Staff
In the case of care staff, GrupaDash2 aims to monitor and optimize the effectiveness of both individual and team activities. This includes:
AI optimization:
GrupaDash3 – Building
In the context of a building, GrupaDash3 is a tool for monitoring and optimizing the functioning of the entire facility, as well as the efficiency of automatic care systems.
Measurement compared to similar objects:
Optimization includes:
Modern mobile devices are becoming increasingly advanced, offering features that support senior care:
Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 on the establishment of harmonised rules on artificial intelligence and amending regulations and directives. This regulation aims to establish uniform regulations on artificial intelligence (AI) in the European Union, in order to ensure safety, transparency and responsibility in the use of AI technologies, and to promote innovation and development in this area. The aforementioned changes also apply to biometric systems, which are becoming increasingly popular in various sectors, including healthcare, security and finance.
Biometric systems are technologies that use unique physical or behavioral characteristics of the user to identify or analyze them. In the context of AI and new regulations, these technologies are used for various purposes, such as security, personalization of services or health diagnostics.
Biometric systems include:
According to Regulation 2024/1689, biometric systems must meet certain principles and standards in terms of privacy protection, personal data security and transparency. Many biometric systems, especially those used for identification purposes, require the user’s consent to process their biometric data. Processing of biometric data must be limited to the data necessary to minimize the risk of privacy breaches. All biometric data must be appropriately protected against unauthorized access or leakage, which involves requirements for data encryption and storage. Users must be informed about how their biometric data is used and stored, as well as about the possibilities of withdrawing consent to their processing.
If an application uses AI to analyze biometric features for the purpose of diagnosing a health condition, it may be considered a high-risk system, which requires assessing the system’s compliance by conducting tests to demonstrate the AI’s safety and effectiveness. In addition, the algorithm must be transparent, with users being informed about how the biometrics work and having the option to disable it. Additionally, decisions made by AI should be subject to the obligation of human supervision, where its actions should not be completely automatic and should be subject to human verification.
Automation of processes in medicine using artificial intelligence (AI) has enormous potential to improve the quality of healthcare and the efficiency of medical personnel. These technologies can change the way we diagnose and monitor diseases, as well as how we organize work in hospitals and clinics.
AI systems that analyze medical images such as X-rays, CT scans, and MRIs can detect abnormalities that the human eye might miss, which is crucial for early diagnosis, especially in diseases such as cancer. AI can help identify changes in patients’ bodies more quickly, which in many cases can save lives.
Thanks to artificial intelligence, bots and patient triage systems can effectively divide patients based on their symptoms, which helps to assign them to the right specialists faster. This not only increases the efficiency of treatment, but also allows for better resource management in hospitals, which is invaluable in times of health crises, when the number of patients can increase rapidly.
AI in cardiology data analysis is especially important in the context of the growing number of heart diseases. AI algorithms can analyze huge amounts of data that are collected from various sources, such as ECGs, blood pressure monitoring, and other medical devices. With this technology, doctors can detect early signs of heart disease faster and take appropriate steps to prevent serious complications, which can improve patient prognosis.
Helping patients monitor their health, especially those with chronic conditions, is another area where AI can provide significant benefits. Through mobile apps, wearables and monitoring systems, patients can track their vital signs in real time, giving them more control over their health and enabling them to respond more quickly to emerging issues.
By automating many processes, medical staff gain time for more complex and demanding interventions. AI helps reduce the time spent on routine tasks, such as analyzing test results or monitoring patient parameters, which means that doctors and nurses can focus on more demanding cases, as well as on direct interaction with patients.
It is not permitted to place on the market AI systems that use subliminal techniques, i.e. techniques that operate outside the person’s awareness, or manipulative or misleading techniques. The purpose of these techniques must not be to significantly change the behaviour of a person who would not be able to make an informed decision, which could lead to serious harm to that person or to other people. AI systems must not exploit the vulnerabilities of people due to their age, disability or particular social or economic situation. Such systems must not be used in a way that leads to the manipulation of the behaviour of a person or group of people, causing serious harm or risk of harm. AI systems must not be used to assess individuals or groups of people based on their social behaviour or other personal characteristics in a way that leads to unfair treatment of individuals or groups in contexts unrelated to the data that were originally used for the assessment, and unjustified or disproportionate treatment of individuals in relation to their social behaviour. AI systems may not be used to draw conclusions about the emotions of individuals in the workplace or in schools, except when used for medical (e.g. psychological diagnosis) or security (e.g. threat assessment) purposes.
These regulations aim to protect people from using AI in ways that could harm their autonomy, dignity and privacy. They prevent the use of technologies that could manipulate people, mislead them or take advantage of them in difficult life situations. Privacy and ethics are key in the context of using AI, especially in areas such as health, education or employment, where decisions about an individual can have serious consequences. These regulations also aim to counter unfair practices, such as social scoring , which can lead to discrimination and unequal treatment of people based on their personal data, emotions or social behavior.
Act of 6 November 2008 on patient rights and the patient’s rights advocate – medical data may only be transferred to persons authorized by the patient, additionally the senior must consent to the application notifying a specific caregiver about their health condition. The notification cannot contain unnecessary medical information to avoid violating privacy.
According to the Act of 14 December 2018 on the protection of personal data processed in connection with the prevention and combating of crime, the processing of sensitive data is prohibited, because it constitutes information that requires special protection due to its sensitivity. However, there are situations in which the processing of sensitive data is permitted. The regulations allow the processing of sensitive data concerning the protection of health or life. Processing may be crucial in the context of using AI in healthcare, but it must be carried out in accordance with the law. Such data may include information about the patient’s health, medical history, biometric or genetic data, which AI can use to diagnose, predict diseases or personalise treatment. Therefore, it is important that AI systems comply with the principles of protecting patient privacy, ensuring compliance with the GDPR and other regulations on the protection of personal data.
Under the provisions of the GDPR, the patient has the right to request the restriction of the processing of his or her personal data, especially when such data is processed in the context of medical records or other sensitive data, based on Article 9 paragraph 2 letter h) of the GDPR. This right gives the patient the possibility to restrict further processing of the data, which may include actions such as:
However, the patient’s right to request restriction of processing is not absolute. There are certain exceptions in which data may be processed even if the patient requests their restriction. The GDPR provides for situations where the restriction of processing could harm important public interests or cause difficulties in fulfilling legal obligations. In such cases, data processing may continue if:
Certain activities in healthcare systems will not be considered “automated decision-making” within the meaning of Article 22 of the GDPR, provided they meet certain conditions. According to these principles, automated processing of personal data can support medical processes, but the final decision regarding the patient’s health must be made by medical personnel and not by artificial intelligence algorithms.
Automatic determination of medical scale results – Algorithms can determine whether a given analysis result is within a specified range (e.g. “within normal”, “above normal”, “below normal”), but the final decision on diagnosis and further treatment remains with the doctor.
Assessment of disease risk based on genome analysis – Systems can analyze a patient’s genetic data and assess the risk of disease, e.g. by identifying genetic mutations, but the decision on further treatment is made by the doctor.
Supporting the therapeutic process – Algorithms can suggest possible drug therapies or diagnostic tests, but the healthcare professional makes the final decision regarding the treatment plan.
Patient selection for preventive and screening tests – Algorithms can help select patients based on defined criteria (e.g. age, gender, medical history), but the decision to qualify for testing must be made by healthcare professionals.
Prescription and follow-up management – AI can remind healthcare professionals to issue a prescription or refer a patient for a follow-up visit, but the doctor makes the final decision.
Preventive health screening and occupational medicine – Algorithms can support decision-making by helping to identify which patients should be referred for specific tests based on work-related factors, but the final decision rests with healthcare professionals.
Medical applications and algorithms as medical devices
If algorithms and applications are part of medical devices that have been approved for marketing in the European Union and certified in accordance with applicable law, they may support diagnostic and therapeutic processes, but the ultimate responsibility for medical decisions remains with the medical staff.
Only the healthcare professional who created the medical records may have access to personal data or individual medical data processed in the service provider’s IT system or in the medical information system: this refers to the healthcare professional who created the electronic medical records containing the personal data or individual medical data of the service recipient.
A medical device is a tool, apparatus, device, software, implant, reagent, material or other article that is intended by the manufacturer for use in humans for at least one of the following medical purposes:
Medical devices do not achieve their primary action by pharmacological, immunological or metabolic means in the human body, but their function may be assisted by such means.
An app can be considered a medical device if it analyzes health data to monitor health and suggests actions or makes decisions that may affect the user’s treatment.
Medical devices are subject to the provisions of the MDR 2017/745 Regulation, are legally obliged to EU Certification, by performing safety tests and also to report to the Office for Registration of Medicinal Products. However, if the application only collects and transfers data to the caregiver, it does not have to be recognized as a medical device
According to Regulation (EU) 2024/1689, products containing digital elements, including artificial intelligence systems, must meet certain cybersecurity requirements, which requires a conformity assessment procedure.
Products with digital elements that are to meet cybersecurity requirements must undergo an appropriate conformity assessment procedure. According to Article 43 of the Regulation, notified bodies responsible for checking the compliance of high-risk AI systems must also supervise compliance with the cybersecurity requirements of these systems.
The regulation provides for some derogations from the standard conformity assessment procedure for products with digital elements, which are subject to special assessment procedures. Although such products may be classified as high-risk AI systems, they still have to undergo the appropriate cybersecurity conformity assessment procedure, as required by the regulation.
The Regulation allows manufacturers of digital products that are subject to cybersecurity requirements to participate in regulatory sandboxes (Article 57 of the Regulation). Regulatory sandboxes are controlled environments where companies can test innovative AI-related solutions while ensuring that their technologies meet all regulatory standards and comply with cybersecurity requirements.
In Poland, there are also regulations such as the Healthcare Information System Act, which specifies the rules for collecting, processing and sharing medical data. This act regulates, among other things, the functioning of electronic medical records and ensures the security of teleinformatic systems used in healthcare. Through the application of these regulations, it is possible to effectively implement AI while maintaining the confidentiality and integrity of patient data.
In addition, the Code of Medical Ethics imposes an obligation to use technology in accordance with the principles of professional ethics. According to Article 57 of the Code, a physician is obliged to use only those diagnostic and therapeutic methods that are consistent with the current state of medical knowledge and are considered safe. The implementation of AI in medicine must therefore take place with respect for the principles of medical ethics, which means, among other things, ensuring human responsibility for decisions made by algorithms and avoiding situations in which the patient would be solely dependent on the action of AI.
Article 32 of the GDPR (General Data Protection Regulation) addresses the issue of security of personal data processing and requires data controllers and data processors to take appropriate measures to ensure an adequate level of protection of data during processing.
The controller and the processor of personal data must implement appropriate technical and organizational measures to ensure an appropriate level of security depending on the type and risk of data processing. Examples of these measures include:
Pseudonymisation and encryption of personal data: Protecting data from access by unauthorized persons through pseudonymisation (e.g. replacing personal data with pseudonyms) and encryption, thereby ensuring their confidentiality and integrity.
Confidentiality, integrity, availability and resilience of systems: Ensuring that data is accessible only to authorized persons and is protected from unauthorized access. It is also about protecting data from corruption, alteration or destruction.
Rapid recovery of data availability in the event of an incident: In the event of a system failure or other incident, the data controller must ensure the ability to quickly recover data and restore its availability to prevent data loss.
Testing, measuring and evaluating effectiveness: Regularly testing, monitoring and evaluating the effectiveness of the security measures implemented to ensure that they are sufficient and effective in ensuring the security of the personal data processed.
The controller and the processor must assess the risks associated with the processing of personal data and implement appropriate security measures that are adequate. Risk of accidental or unlawful: destruction, loss, modification, unauthorized disclosure or unauthorized access to personal data. Data controllers must take into account such risks and appropriately select technical and organizational measures that will protect data against these risks.
Article 54 of Regulation (EU) 2024/1689, in relation to a product with digital elements posing a risk in cyberspace, provides for the following actions and procedures: If a market surveillance authority finds that a product with digital elements poses a significant risk, it must inform the relevant notified body. Market surveillance authorities are required to act in accordance with the procedures in Article 18 of Regulation (EU) 2019/1020, which covers issues related to market surveillance and ensuring product compliance.
Market surveillance authorities must take into account not only technical but also non-technical risk factors when assessing products. Such factors include, among others, issues related to the security of critical supply chains, which have been assessed at EU level under Article 22 of Directive (EU) 2022/2555.
In the event of risks related to these factors, the market surveillance authority shall inform the relevant authorities designated under Article 8 of Directive (EU) 2022/2555 and cooperate with them in order to reduce the risk.
If the market surveillance authority finds that non-compliance is not limited to the territory of its Member State, it is obliged to inform the Commission and the other Member States of the results of the assessment carried out and the actions taken against the manufacturer of the product. This cooperation mechanism ensures consistency of action throughout the European Union.
Economic operators who place products with digital elements on the EU market are required to take appropriate corrective measures to ensure compliance with security and cybersecurity regulations. This obligation applies to products available on the market throughout the European Union.
If the manufacturer does not take corrective action within the deadline, the market surveillance authority has the possibility to impose provisional measures, which may include: Prohibiting or restricting the product being made available on the market, withdrawing the product from the market, recalling the product. Such measures must be notified to the Commission and the other Member States without delay, in order to ensure consistency and response across the Union.
Where provisional measures are imposed, the market surveillance authority must provide detailed information on the product, including:
If other Member States have taken measures in relation to the product in question, although they were not responsible for initiating the procedure, they must inform the Commission and the other Member States of the action taken. They may also provide additional information on the non-compliance of the product or express reservations about the action taken.
Economic operators responsible for the device must cooperate with the competent authorities of the Member States in order to carry out this evaluation and to resolve any potential issues related to the safety or compliance of the device.
Article 95 describes the procedure to be followed in the case of products presenting unacceptable risks to health and safety.
Following a conformity assessment of a device (pursuant to Article 94), if the competent authorities find that the device presents an unacceptable risk to health, the safety of patients, users or other persons, or public health, they must immediately require the manufacturer and other relevant economic operators to take corrective actions. These actions include:
The competent authorities shall immediately notify:
Notification shall be made via the electronic system (referred to in Article 100) and shall include information on:
Economic operators must immediately take appropriate corrective action on the EU-wide market in respect of all products they have made available.
If the economic operator does not take adequate corrective action within the deadline, the competent authorities shall take appropriate measures to:
Again, these measures are transmitted via an electronic system to the Commission, the other Member States and the notified body.
This notification should include all available information, such as:
Other Member States that are not the Member State initiating the procedure must also use the electronic system to:
Where, within two months of receipt of the notification, no objections are raised by either a Member State or the Commission against the measures taken by a Member State, the measures are deemed justified. In such a case, all Member States must immediately take appropriate measures restricting or prohibiting the availability of the product on their national market, including withdrawing the product from the market or recalling it.
Artificial intelligence has enormous potential to revolutionize medicine. In the coming years, we can expect further development of advanced diagnostic algorithms. Its ability to analyze data, precise diagnostics and support doctors in making decisions means that medicine is entering a new era of innovation. And the SeniorApp.pl application is part of this, because through the AI software used in it, it will not only increase the possibility of daily monitoring of seniors’ health parameters, but also facilitate the work of medical staff and relieve them of their duties. However, for this technology to be fully utilized, it is necessary to develop appropriate regulations and ensure patient safety.