What is robots.txt?

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.
In practice, robots.txt files indicate whether certain user agents (web-crawling software) can or cannot crawl parts of a website.

Related terms (by category)

YMYL (Your Money or Your Life)

It refers to content that can significantly impact a person’s health, financial stability, safety, or overall well-being

AIO (AI Overviews)

AI Overviews are a brand new (introduced in Google I/O 2024) core Google Search feature, like knowledge panels.

Knowledge Graph

Google’s search results often display information sourced from the Knowledge Graph, a vast database containing billions of facts about people, places, and things.

knowledge panels

Knowledge panels are information boxes that show up on Google when you search for specific entities like people, places, organizations, or things that are part of the Knowledge Graph.

GEO (Generative Engine Optimization)

GEO involves optimizing content specifically for AI-driven search engines, known as generative engines

Parasite SEO

Parasite SEO is a strategy where marketers leverage the authority and ranking power of high-traffic websites to promote their content or links.

Related terms (by alphabet)

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.

Necessary

Always Enabled

Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.

Cookie	Duration	Description
_ga	2 years	The _ga cookie, installed by Google Analytics, calculates visitor, session and campaign data and also keeps track of site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognize unique visitors.
_gat_UA-52042964-1	1 minute	A variation of the _gat cookie set by Google Analytics and Google Tag Manager to allow website owners to track visitor behaviour and measure site performance. The pattern element in the name contains the unique identity number of the account or website it relates to.
_gcl_au	3 months	Provided by Google Tag Manager to experiment advertisement efficiency of websites using their services.
_gid	1 day	Installed by Google Analytics, _gid cookie stores information on how visitors use a website, while also creating an analytics report of the website's performance. Some of the data that are collected include the number of visitors, their source, and the pages they visit anonymously.
_lfa	2 years	This cookie is set by the provider Leadfeeder to identify the IP address of devices visiting the website, in order to retarget multiple users routing from the same IP address.
apbct_cookies_test	session	CleanTalk sets this cookie to prevent spam on comments and forms and act as a complete anti-spam solution and firewall for the site.
apbct_page_hits	session	CleanTalk sets this cookie to prevent spam on comments and forms and act as a complete anti-spam solution and firewall for the site.
apbct_prev_referer	session	Functional cookie placed by CleanTalk Spam Protect to store referring IDs and prevent unauthorized spam from being sent from the website.
apbct_site_landing_ts	session	CleanTalk sets this cookie to prevent spam on comments and forms and act as a complete anti-spam solution and firewall for the site.
apbct_site_referer	3 days	This cookie is placed by CleanTalk Spam Protect to prevent spam and to store the referrer page address which led the user to the website.
apbct_timestamp	session	CleanTalk sets this cookie to prevent spam on comments and forms and act as a complete anti-spam solution and firewall for the site.
apbct_urls	3 days	This cookie is placed by CleanTalk Spam Protect to prevent spam and to store the addresses (urls) visited on the website.
CONSENT	2 years	YouTube sets this cookie via embedded youtube-videos and registers anonymous statistical data.
cookielawinfo-checkbox-advertisement	1 year	Set by the GDPR Cookie Consent plugin, this cookie is used to record the user consent for the cookies in the "Advertisement" category .
cookielawinfo-checkbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
CookieLawInfoConsent	1 year	Records the default button state of the corresponding category & the status of CCPA. It works only in coordination with the primary cookie.
ct_checkjs	session	CleanTalk–Used to prevent spam on our comments and forms and acts as a complete anti-spam solution and firewall for this site.
ct_fkp_timestamp	session	CleanTalk sets this cookie to prevent spam on the site's comments/forms, and to act as a complete anti-spam solution and firewall for the site.
ct_pointer_data	session	CleanTalk sets this cookie to prevent spam on the site's comments/forms, and to act as a complete anti-spam solution and firewall for the site.
ct_ps_timestamp	session	CleanTalk sets this cookie to prevent spam on the site's comments/forms, and to act as a complete anti-spam solution and firewall for the site.
ct_timezone	session	CleanTalk–Used to prevent spam on our comments and forms and acts as a complete anti-spam solution and firewall for this site.
elementor	never	This cookie is used by the website's WordPress theme. It allows the website owner to implement or change the website's content in real-time.
PHPSESSID	session	This cookie is native to PHP applications. The cookie is used to store and identify a users' unique session ID for the purpose of managing user session on the website. The cookie is a session cookies and is deleted when all the browser windows are closed.
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
vuid	2 years	Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos to the website.

Functional

Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.

Cookie	Duration	Description
_gr	2 years	This cookie captures the email of the user when identified. We have three (3) ways to identify the email of the user. a) when user clicks on a link of a Growthrocks' campaign, b) when user is logged-in and c) when a user submits a form containing an email
_gr_flag	2 years	A simple cookie to capture that information of the user has been sent to Marketing Automation. If true no further information will be sent. If no, the system will try to send information when the email will be filled
bcookie	2 years	LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser ID.
bscookie	2 years	LinkedIn sets this cookie to store performed actions on the website.
lang	session	LinkedIn sets this cookie to remember a user's language setting.
lidc	1 day	LinkedIn sets the lidc cookie to facilitate data center selection.
UserMatchHistory	1 month	LinkedIn sets this cookie for LinkedIn Ads ID syncing.

Performance

Analytics

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.

Cookie	Duration	Description
_ga	2 years	The _ga cookie, installed by Google Analytics, calculates visitor, session and campaign data and also keeps track of site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognize unique visitors.
_gat_UA-52042964-1	1 minute	A variation of the _gat cookie set by Google Analytics and Google Tag Manager to allow website owners to track visitor behaviour and measure site performance. The pattern element in the name contains the unique identity number of the account or website it relates to.

Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.

Cookie	Duration	Description
__Host-GAPS	2 years	This cookie allows the website to identify a user and provide enhanced functionality and personalisation.
_fbp	3 months	This cookie is set by Facebook to display advertisements when either on Facebook or on a digital platform powered by Facebook advertising, after visiting the website.
fr	3 months	Facebook sets this cookie to show relevant advertisements to users by tracking user behaviour across the web, on sites that have Facebook pixel or Facebook social plugin.
test_cookie	15 minutes	The test_cookie is set by doubleclick.net and is used to determine if the user's browser supports cookies.
VISITOR_INFO1_LIVE	5 months 27 days	A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.
YSC	session	YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.
yt-remote-connected-devices	never	YouTube sets this cookie to store the video preferences of the user using embedded YouTube video.
yt-remote-device-id	never	YouTube sets this cookie to store the video preferences of the user using embedded YouTube video.

Others

Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.

Cookie	Duration	Description
_lfa_test_cookie_stored	past	No description
AnalyticsSyncHistory	1 month	No description
apbct_visible_fields_0	session	No description
apbct_visible_fields_1	session	No description
apbct_visible_fields_2	session	No description
apbct_visible_fields_3	session	No description
apbct_visible_fields_4	session	No description
apbct_visible_fields_5	session	No description
apbct_visible_fields_6	session	No description
apbct_visible_fields_7	session	No description
ct_checked_emails	session	No description
ct_has_scrolled	session	No description
ct_mouse_moved	session	No description
ct_screen_info	session	No description
GetLocalTimeZone	session	No description
ifso_last_viewed	session	No description available.
ifso_visit_counts	1 year	No description available.
li_gc	2 years	No description
nitroCachedPage	session	No description
prism_649664625	1 month	No description

Related terms (by category)

Related terms (by alphabet)

About

Services

Special Cases

Industries

Latest Posts

Call us

Get actionable growth tips

Have a cookie :)