Generative AI models, while primarily designed for benign purposes, present potential dual-use risks as they can be repurposed by attackers to assist in certain phases of cyberattacks, though they lack the autonomy and intent to carry out sophisticated attacks independently. These models, such as GPT-4, have evolved significantly, possessing a vast number of parameters that parallel the human brain's synapses, sparking concerns over their misuse in cybersecurity. While there are no widely publicized cases of generative AI being used to execute cyberattacks, the technology's ability to generate realistic media and simulate personas could facilitate phishing and social engineering. Experiments reveal that AI, like ChatGPT, can provide insights for some attack steps but often fails to produce useful malicious code, highlighting its current limitations. Conversely, generative AI holds promise for enhancing cybersecurity defense by improving comprehension, guiding incident response, and boosting human capabilities. Ultimately, the responsibility to use AI technologies ethically lies with humans, as these tools remain enablers rather than autonomous agents in both offensive and defensive contexts.